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Abstract 

Noncontact  measurements  of  lightweight  flexible  aerospace  structures  present  sev¬ 
eral  challenges.  Objects  are  usually  mounted  on  a  test  stand  because  current  noncontact 
measurement  techniques  require  that  the  net  motion  of  the  object  be  zero.  However, 
it  is  often  desirable  to  take  measurements  of  the  object  under  operational  conditions, 
and  in  the  case  of  miniature  aerial  vehicles  (MAVs)  and  deploying  space  structures,  the 
test  article  will  undergo  significant  translational  motion.  This  thesis  describes  a  hybrid 
noncontact  measurement  system  which  will  enable  measurement  of  structural  kinemat¬ 
ics  of  an  object  freely  moving  about  a  volume.  By  using  a  real-time  videogrammetry 
system,  a  set  of  pan-tilt-zoom  (PTZ)  cameras  is  coordinated  to  track  large-scale  net 
motion  and  produce  high-speed,  high-quality  images  for  photogrammetric  surface  recon¬ 
struction.  The  design  of  the  system  is  presented  in  detail.  A  method  of  generating  the 
calibration  parameters  for  the  PTZ  cameras  is  presented  and  evaluated  and  is  shown  to 
produce  good  results.  The  results  of  camera  synchronization  tests  and  tracking  accu¬ 
racy  evaluation  are  presented  as  well.  Finally,  a  demonstration  of  the  hybrid  system  is 
presented  in  which  all  four  PTZ  cameras  track  an  MAY  in  flight. 
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A  Photogrammetry-based  Hybrid  System  for  Dynamic 


Tracking  and  Measurement 

I.  Introduction 

Lightweight  membranes  and  structures  are  drawing  increased  interest  for  use  in 
a  variety  of  applications.  Membranes  and  inflatable  structures  are  part  of  many 
proposed  satellite  concepts  serving  a  variety  of  purposes  such  as  antenna,  solar  sails,  and 
heat  shields.  These  materials  are  intended  to  be  compactly  stored  for  launch  and  then 
unfurl  once  in  space.  When  fully  deployed,  the  structures  will  need  to  be  able  to  resist 
disturbance  caused  by  accelerations  of  the  spacecraft,  electromagnetic  momentum,  and 
other  events.  Miniature  aerial  vehicles  (MAVs)  are  a  new  class  of  flight  vehicles  with 
dimensions  typically  less  than  1  m.  Because  of  their  low  Reynolds  number  flight  regime, 
MAVs  require  a  novel  design  approach.  Lightweight,  flexible  material  is  now  not  only 
tolerated  but  also  designed  into  the  aircraft  and  plays  a  key  role  the  in  the  achieving 
the  desired  flight  characteristics  of  the  vehicle.  The  prevalence  of  flexible  and  flapping 
wings  on  flying  animals  illustrate  their  usefulness  and  possible  optimality  for  flight  at 
this  scale. 

For  effective  design  and  operation  of  satellites  and  MAVs,  measurements  must  be 
taken  of  their  response  under  normal  operating  conditions  and  anomalous  disturbances. 
A  satellite  operator  must  be  able  to  tell  whether  his  deployable  structure  has  deployed 
correctly.  Resonant  frequencies  of  the  structures  must  be  known  and  designed  around. 
Effects  such  as  wrinkling  must  be  detected,  as  they  can  affect  structure  performance. 
MAV  designers  are  still  wrestling  with  the  generation  of  accurate  computer  models  for 
flight  at  Reynolds  number  on  the  order  of  104  and  lower,  and  require  flight  data  to 
validate  these  models.  However,  lightweight  structures  pose  significant  measurement 
challenges.  Large  membranes  have  large  displacement,  long-period  dynamics  that  are 
difficult  to  measure  with  traditional  equipment  such  as  strain  gages.  Small  structures 
are  of  comparable  weight  to  the  gages  themselves,  and  attaching  them  to  the  structure 
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affects  the  dynamic  response.  Adding  sensors  to  MAVs  for  in-flight  testing  is  impractical 
because  of  payload  weight  limits. 

Noncontact  measurement  methods  are  a  logical  solution  to  the  measurement  diffi¬ 
culties  described  above.  There  are  many  noncontact  measurement  methods  in  use  today. 
Laser  vibrometry,  the  measurement  of  velocity  of  an  object  based  the  interference  of  the 
a  laser  beam  with  the  reflected  response  from  an  object,  is  the  standard  for  dynamic 
measurements,  boasting  high  spacial  resolution  and  sampling  frequency.  Photogramme- 
try  is  the  process  of  deriving  the  three-dimensional  coordinates  of  object  points  based  on 
their  two  dimensional  location  in  a  set  of  photographic  images,  and  is  increasingly  in  use 
as  computer  processing  and  digital  imaging  technology  matures.  Other  methods  such  as 
laser  range  finders,  capacitance  measurement,  and  various  interferometry  techniques  are 
also  found  in  the  literature.  These  techniques  are  discussed  in  greater  detail  in  the  next 
chapter.  All  of  these  techniques  have  their  strengths  and  weaknesses,  and  none  of  them 
represent  a  complete  solution  to  the  problem  of  membrane  measurement.  Laser  vibrom¬ 
etry  has  difficulty  with  large  displacement  of  large  membrane  structures  and  the  high 
sampling  frequency  would  be  unnecessary.  Capacitance  and  interferometry  techniques 
require  very  flat,  and  in  the  case  of  capacitance,  conductive,  material.  The  accuracy  of 
photogrammetry  methods  are  limited  by  camera  resolution,  and  processing  can  often  be 
labor  intensive.  The  optical  characteristics  of  the  material  itself,  such  as  reflectance  or 
transparency,  can  make  measurement  difficult.  All  of  the  above  measurement  techniques 
would  be  difficult  to  use  on  a  dynamic  object  such  as  a  MAV  in  flight  or  an  extending 
satellite  boom. 

Therefore,  new  measurement  techniques  must  be  developed  confront  these  lim¬ 
itations.  Many  of  the  limitations  can  be  solved  with  creative  application  of  current 
techniques.  For  instance,  recent  research  has  enabled  the  use  of  photogrammetry  on 
transparent  membranes  [1].  Also,  combining  data  from  different  sensors  has  been  inves¬ 
tigated  to  overcome  the  limitations  of  the  individual  measurement  systems  [2], 

This  thesis  proposes  a  solution  to  one  lightweight  structures  measurement  problem 
in  particular:  Surface  reconstruction  of  an  object  that  is  dynamically  moving  throughout 
a  volume.  Examples  include  measurement  of  the  wing  shape  of  a  flapping  MAV  in  free 
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flight,  wing  flex  on  a  fixed-wing  MAV,  deployment  of  a  solar  sail,  and  deployment  of  a 
boom.  A  hybrid  measurement  system  approach  is  taken,  in  which  large-scale  motion  of 
the  object  is  measured  by  a  real-time  videogrammetry  system  and  used  to  direct  four 
pan-tilt-zoom  (PTZ)  cameras  for  taking  high-resolution  images  of  the  area  of  interest. 
The  image  data  is  then  processed  with  photogrammetry  software  with  which  a  dense 
surface  profile  is  generated.  The  work  of  this  thesis  was  to  build  this  system,  test  and 
evaluate  its  capabilities,  and  establish  that  this  approach  will  produce  effective  image 
data  and  system  state  information  for  the  accurate  reconstruction  of  lightweight  dynamic 
structures. 

This  document  is  arranged  as  follows:  Chapter  2  presents  the  review  of  current 
lightweight  structures  measurement  research,  with  a  particular  focus  on  photogrammetry. 
Chapter  3  describes  in  detail  the  components  of  the  measurement  system,  their  operation 
and  integration.  Chapter  4  presents  the  tests  performed  to  validate  the  system  and 
determine  its  capabilities,  and  the  results  of  the  tests.  Chapter  5  presents  the  conclusions 
drawn  from  the  research  and  suggests  directions  for  future  work. 
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II.  Background 


With  the  increased  use  of  and  interest  in  lightweight  structures,  there  has  been  a 
significant  amount  of  research  undertaken  into  the  development  of  noncontact 
shape  measurement  systems.  These  measurement  systems  vary  greatly  in  accuracy,  ease 
of  use,  and  adaptability,  and  so  the  choice  of  measurement  system  is  highly  application- 
dependent.  Many  noncontact  measurement  systems  have  been  investigated  for  their 
suitability  for  the  measurement  of  miniature  aerial  vehicle  wing  surface  profiles  and 
lightweight  space  structures.  This  chapter  presents  an  overview  of  the  current  state  of 
the  art  in  noncontact  measurement  systems  as  they  relate  to  these  applications. 

Methods  that  have  gained  considerable  attention  recently  are  photogrammetry  and 
its  relative,  videogrammetry.  These  methods  form  the  basis  of  the  hybrid  measurement 
system  that  is  the  work  of  this  thesis,  and  will  be  discussed  in  detail.  Additional  methods 
such  as  laser  vibrometry  and  range  hireling,  Moire  fringe  projection  and  capacitance  are 
also  found  in  the  literature  and  these  are  compared  to  photogrammetry  techniques. 
Finally,  research  into  hybrid  systems  including  two  or  more  measurement  techniques  is 
presented. 

2. 1  Photogrammetry 

Photogrammetry  is  the  process  of  deriving  the  three-dimensional  coordinates  of 
object  points  based  on  their  two  dimensional  location  in  a  set  of  photographic  images. 
Photogrammetry  has  its  origins  in  the  mid-1800s,  following  immediately  the  invention  of 
direct  photography  by  Louis  Daguerre  [3].  Photogrammetry  developed  gradually  through 
the  early  part  of  the  20th  century,  due  to  the  difficulty  of  working  with  photographic  film, 
but  still  many  high  precision  analog  systems  were  developed  for  use  in  surveying  and 
map  making,  along  with  the  mathematical  principles  of  photogrammetric  triangulation 
and  error  evaluation.  The  last  30  years  have  seen  the  advent  of  inexpensive  digital 
computing  and  photography,  as  well  as  advances  in  optics,  which  has  allowed  greater 
versatility  and  automation  of  the  photogrammetric  process  [4].  Photogrammetry  is  now 
used  for  applications  as  varied  as  robot  vision,  as-built  facilities  measurement,  digitization 
of  cultural  artifacts,  and  biometrics  analysis  [5]. 
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Photogrammetry  techniques  can  be  divided  into  two  main  categories:  target-based 
and  texture-based.  Target-based  photogrammetry  requires  the  use  of  physical  or  pro¬ 
jected  marks  on  an  object  to  aid  in  point  referencing  between  images.  The  targets  are 
designed  to  appear  as  high  contrast  points  in  the  images,  and  so  are  easy  for  imaging 
software  to  identify.  To  further  automate  processing,  some  systems  use  coded  targets 
which  can  be  directly  referenced  between  images.  Texture-based  photogrammetry  uses 
natural  or  projected  texture  of  the  object  to  correlate  points  between  a  set  of  images. 
This  process  produces  a  denser  set  of  point  data  than  target-based  methods,  but  is 
less  automated  and  more  object-dependent.  In  order  to  be  suitable  for  texture-based 
methods,  the  surface  of  the  object  must  have  sufficient  contrast  variation  to  be  identifi¬ 
able  to  the  correlation  software.  Furthermore,  the  image  angle  plays  a  large  role  in  the 
identification  and  correlation  of  points  between  images  [6]. 

In  either  case,  the  fundamental  problem  of  photogrammetry  is  to  determine  the 
relationship  between  the  photographed  objects  and  the  captured  image.  The  captured 
image  is  the  result  of  collecting  and  recording  electromagnetic  radiation  reflected  (or 
emitted)  by  the  objects.  Therefore,  the  relationship  between  image  and  objects  should 
describe  the  path  of  rays  between  the  object  and  imaging  sensor.  The  relationship  is  de¬ 
scribed  by  a  camera  model  and  an  atmospheric  model.  For  close  range  photogrammetry, 
the  atmosphere  is  assumed  to  have  a  negligible  effect  on  the  path  of  the  rays,  and  thus 
the  relationship  is  completely  described  by  the  camera  model  [3].  The  development  of 
the  camera  model  is  presented  in  the  next  section. 

2.1.1  Pinhole  camera  model.  Most  camera  models  are  based  on  an  ideal  pinhole 
camera.  A  schematic  of  a  pinhole  camera  is  shown  in  Figure  2.1.  A  pinhole  camera  has 
no  lens,  rather  light  passes  through  a  point  (the  pinhole)  in  the  front  of  the  camera  and 
illuminates  the  imaging  plane.  By  passing  all  rays  through  the  same  point,  each  position 
on  the  screen  is  illuminated  by  rays  from  only  one  direction.  The  pinhole  is  called  the 
focal  point  or  perspective  center.  For  the  theoretical  model,  the  imaging  plane  can  be 
moved  in  front  of  the  focal  point  so  that  the  image  appears  as  it  would  in  a  photographic 
print.  The  line  normal  to  the  imaging  plane  that  passes  through  the  focal  point  is  called 
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the  optical  axis,  and  the  point  at  which  it  intersects  the  imaging  plane  is  the  principle 
point.  The  distance  along  the  optical  axis  between  the  imaging  plane  and  the  focal  point 
is  the  focal  length  or  principal  distance. 

The  camera  imaging  process  can  be  described  mathematically  as  the  perspective 
projection  of  three-dimensional  points  onto  a  two-dimensional  plane.  Consider  the  imag¬ 
ing  plane  and  the  three-dimensional  world  points  to  be  represented  by  two  coordinate 
systems  whose  origins  are  at  the  focal  point.  The  imaging  plane  coordinate  system  (ICS) 
x  and  y  axes  lie  in  a  plane  parallel  to  the  image  plane  with  the  x  axis  pointing  towards 
right  side  of  the  image  plane  and  the  y  axis  pointing  towards  the  top.  The  z  axis  is 
aligned  with  the  optical  axis.  The  global  coordinate  system  (GCS)  can  have  any  orien¬ 
tation.  The  transformation  of  a  point  [XYZ]  in  GCS  to  point  [xyz]  in  ICS  is  described 
by 

X 

M  Y 
Z 

where  M  is  a  3x3  rotation  matrix  defined  by  Euler  angles  (f),  6 ,  and  ^  that  rotates  GCS 
into  ICS,  and  k  is  a  scale  factor  between  the  coordinate  systems.  The  z  coordinate  of  all 
points  in  the  image  plane  is  equal  to  the  negative  of  the  focal  length,  and  since  the  GCS 
origin  does  not  typically  coincide  with  the  focal  point,  translational  terms  Xl,  Yl,  and 
ZL  are  introduced 

X  -  XL  x 

M  Y-Yl  =1  y  (2.2) 

z~Zl\  [-/ 

The  values  of  Xl,  Yl ,  and  Zl  and  (j),  9,  and  ^  are  known  as  external  camera  parameters 
[4],  If  matrix  M  is  made  up  of  elements  rn,h3 ,  Equation  2.2  can  be  rewritten  as  three 
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Camera  Image  Theoretical 


Figure  2.1: 

equations 

x  =  k[m,iti(  X 
y  =  k[m2,i(X 
~f  =  k[m3>  i(X 


Schematic  of  a  pinhole  camera. 


XL)  +  m1)2(y  -  Yl)  +  mh3(Z  -  ZL )] 
XL)  +  m2>2(y  -  Yl)  +  m2,3(Z  -  ZL )] 
^l)  +  m3>2(y  -  Yl)  +  m3,3(y  -  ZL)\ 


and  eliminating  scale  factor  k  leaves 


mhl(X~XL)  +  m1}2(Y 
m3t i(X  -  XL)  +  m3,2(y 
_  m2;i(X  -  XL)  +  m2[2(y 
V~  :m3>1(X-XL)  +  m3}2(Y 


YL)+mlt3(Z-ZL) 
Yl)  +  m3y3(Z  -  ZL) 
Yl)  +  m2t3(Z  -  ZL) 
Yl)  +  m3t3(Z  —  ZL) 


(2.3) 

(2.4) 

(2.5) 


(2.6) 

(2.7) 


Equations  2.6  and  2.7  are  called  the  collinearity  equations,  referring  to  the  fact  that  the 
object  point,  focal  point  and  image  point  lie  along  the  same  line.  These  equations  form 
the  basis  of  most  photogrammetry  techniques,  and  can  be  used  to  solve  several  types  of 
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problems  depending  on  which  variables  are  considered  known  and  unknowns.  Unknowns 
are  solved  for  using  an  optimization  routine,  which  is  known  as  bundle  adjustment. 

3-D  point  determination  can  be  performed  with  as  few  as  two  cameras,  but  using 
additional  cameras  allows  for  self-validation  of  the  point  location  results.  The  bundle 
adjustment  produces  point  locations  that  are  the  optimal  solution  to  the  collinearity 
equation  system  based  on  the  image  points,  and  statistical  evaluation  of  the  solution  can 
determine  precision  confidence.  The  optimal  solution  points  do  not  coincide  exactly  with 
the  location  calculated  from  each  photo,  and  the  residual  error  between  image  points  and 
solution  points  is  used  to  calculate  confidence  in  the  point  locations.  Precision  estimation 
is  a  useful  feature  of  photogrammetry  techniques. 

2.1.2  Extended  camera  model.  The  ideal  camera  model  assumes  a  perfect 
projection  from  the  object  point  to  the  image  plane;  however,  in  reality  this  is  not 
the  case.  Causes  of  error  in  the  projection  include  lens  distortion,  misalignment  of  the 
lens,  lens  imperfections,  and  non-flat  image  sensor.  If  the  effects  of  these  flaws  can 
be  modeled  then  the  ideal  camera  model  can  be  modified  to  produce  more  accurate 
results.  Commonly  modeled  error  effects  in  the  literature  are  principle  point  shift,  radial 
distortion,  and  decentering  distortion  [4,  7].  The  values  that  define  the  modeling  of 
these  effects,  along  with  the  focal  length,  are  called  internal  camera  parameters  and  are 
generally  solved  for  with  a  calibration  method.  It  is  important  to  note  that  internal 
camera  parameters  are  often  highly  dependent  on  the  lens  settings  such  as  zoom  and 
focus. 

Principal  point  shift  is  caused  by  a  misalignment  of  the  lens  with  respect  to  the 
image  plane.  The  other  distortion  parameters  are  measured  with  respect  to  the  location 
of  the  principal  point,  and  so  it  must  be  accurately  known  in  order  to  achieve  optimum 
model  results.  However,  adjusting  other  internal  and  external  parameters  to  compensate 
for  an  unknown  principal  point  location  has  been  shown  to  produce  acceptable  results 
[8] .  The  principal  point  shift  is  accounted  for  by  replacing  the  x  and  y  coordinates  with 
adjusted  values  x  —  x0  and  y  —  y0,  where  x0  and  y0  are  the  location  of  the  principal  point. 


Radial  distortion  is  an  increase  or  decrease  in  magnification  based  on  radial  distance 
from  the  principal  point.  It  contributes  to  what  is  know  as  “barrel”  and  “pincushion” 
effects  shown  in  Figure  2.2.  Radial  distortion  can  be  modeled  as  a  polynomial  function 
of  the  radial  distance  r 


dx  =  kixr 2  +  k2xr 4 

(2,8) 

dy  =  kiyr 2  +  k^r4 

(2,9) 

2  2,2 
r  =  x  +  y 

(2.10) 

where  internal  camera  parameters  k\  and  k2  define  the  extent  of  the  distortion.  For  wide 
angle  lenses  it  may  be  necessary  to  extend  the  radial  distortion  polynomial  with  a  sixth 
order  term  [9] .  Other  formulations  of  the  distortion  polynomial  include  using  odd  powers 
of  radial  distance  instead  of  even  [5]. 

Decentering  distortion  is  caused  by  misalignment  of  the  lens.  An  example  of  its 
effect  on  image  points  can  be  seen  in  Figure  2.3.  It  can  be  modeled  and  corrected  for  as 
shown  in  Equation  2.11  and  2.12 

dx  =  pi(r2  +  2x2)  +  2p2xy  (2.11) 

d.y  =  p-2(r2  +  2y2)  +  2pixy  (2.12) 


where  internal  camera  parameters  p\  and  p2  define  the  extent  of  the  distortion.  Decen¬ 
tering  distortion  is  a  less  significant  effect,  being  usually  an  order  of  magnitude  less  than 
radial  distortion  [10]. 

The  extended  model  can  be  written  as 


1 

1 

* 

ti 

1 

x  —  xq  +  dx 

Y  -  Yl 

“  k 

y-yo  +  dy 

z-zL 

- f 

(2.13) 
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Figure  2.2:  Effect  of  pincushion  (a)  and  barrel  (b)  radial  distortion  on  image  points. 
Crosses  represent  undistorted  points  and  circles  show  the  effect  of  distortion 
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Figure  2.3:  Effect  of  decentering  distortion  on  image  points.  Crosses  represent  undis¬ 
torted  points  and  circles  show  the  effect  of  distortion 
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where 


dx  =  ki(x  -  x0)r2  +  k2{x  -  x0 )r4  +  pi(r2  +  2(x  -  x0)2)  +  2p2(^  -  x0)(y  -  yo)  (2-14) 
dy  =  ki (y  -  y0)r 2  +  k2(y  -  y0)r4  +  p2[r2  +  2 (y  -  y0 )2)  +  2^(0;  -  x0)(y  -  y0)  (2.15) 

Other  distortion  parameters  can  be  found  in  the  literature  as  well.  Gruen  [5] 
includes  scaling  and  shear  corrections  to  compensate  for  imprecisions  introduced  by  the 
image  sensor  and  image  acquisition  technique;  however  Remondino  and  Fraser  [11]  state 
that  such  corrections  are  rarely  significant  in  modern  digital  cameras. 

2.1.3  Camera  calibration.  A  calibration  procedure  is  performed  to  determine 
the  internal  camera  parameters  discussed  above,  and  there  are  several  methods.  Images 
can  be  taken  of  a  regularly  spaced  point  cloud  such  as  a  calibration-grid,  and  the  cam¬ 
era  parameters  are  solved  for  along  with  the  point  locations  during  bundle  adjustment. 
This  method  does  not  require  the  grid  measurements  to  be  known  beforehand,  but  does 
need  to  be  performed  separately  before  beginning  a  measurement  project.  Alternatively, 
calibration  can  be  performed  simultaneously  with  project  data  generation.  The  compu¬ 
tational  process  is  identical  to  the  previous  method,  but  project  points  and  image  angles 
are  constrained  by  the  requirements  of  data  collection  and  the  project  environment,  and 
this  can  lead  to  less  robust  configuration  for  the  bundle  adjustment  process.  Remondino 
and  Fraser  [11]  provide  a  good  discussion  of  the  relative  merits  of  different  calibration 
techniques. 

Because  of  the  convenience  and  usefulness  of  zoom  lenses,  a  significant  amount  of 
research  has  gone  into  methods  of  calibration  for  these  systems.  The  intrinsic  camera 
parameters  are  dependent  on  lens  settings  on  focal  length  and  focus,  and  so  means 
of  determining  the  parameters  are  needed  that  are  practical  and  produce  satisfactory 
results.  Willson  [12]  provides  an  in-depth  look  at  pan-tilt-zoom  camera  calibration,  and 
suggests  a  bivariate  polynomial  interpolation  of  the  focal  length,  principal  point  and  k\ 
distortion  parameter  with  respect  to  zoom  and  focus  settings,  based  on  sets  of  calibration 
data  at  many  lens  positions.  More  recently,  Fraser  and  Al-Ajlouni  [10]  described  a  more 
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practical  method  for  use  with  inexpensive  digital  cameras.  Their  results  showed  a  linear 
interpolation  of  the  photogrammetric  focal  length,  principal  point,  and  k\  distortion 
parameter  based  on  the  camera  focal  length  setting  was  sufficient  for  object  accuracy  on 
the  order  of  1:10,000  (one  unit  error  for  10,000  units  object  length). 

Fraser  and  Al-Ajlouni  describe  the  convenience  of  using  zoom  lenses  for  taking 
photogrammetrically  suitable  images  of  architectural  and  historical  sites,  traffic  accidents 
and  other  applications.  The  measurement  system  proposed  in  this  thesis  makes  use  of 
zoom  lenses  for  keeping  the  test  object  well  framed  in  each  image  as  it  moves  around 
the  capture  volume.  A  similar  approach  to  those  taken  in  the  papers  described  above 
will  be  used  to  generate  a  continuum  of  calibration  parameters  based  on  the  settings  of 
the  lenses  at  the  time  each  image  is  taken.  The  use  of  curve  fit  calibration  parameters 
for  processing  a  sequence  of  images  taken  at  changing  lens  settings  has  not  been  found 
in  the  literature  by  this  author. 

2.1.4  Videogrammetry.  Videogrammetry  operates  on  the  same  principles  as 
photogrammetry,  but  applies  to  a  series  of  image  sets.  The  result  is  a  time  history 
of  three  dimensional  point  data  from  which  kinematic  or  vibrational  information  can 
be  extracted.  Because  of  the  number  of  image  sets  to  be  processed,  videogrammetry 
systems  will  often  include  methods  that  reduce  processing  time  and  increase  automation, 
especially  in  the  process  of  feature  recognition.  Processing  using  texture-based  methods 
usually  requires  intelligent  application  of  filters  and  constraints  or  precise  test  setup, 
and  therefore  can  be  very  labor  intensive.  Discrete  targets  are  often  employed  instead  to 
allow  automatic  marking  and  referencing  of  points  between  images  and  epochs  [13-17]. 
Further  robustness  can  be  achieved  by  using  a  light  source,  cameras  filtered  for  a  specific 
wavelength,  and  retroreflective  markers  [18].  The  markers  produce  high  contrast  points 
in  images  which  lend  themselves  to  filtering  of  extraneous  data. 

Real-time  videogrammetry  systems  are  the  extreme  case  of  automated  photogram¬ 
metry  processing.  Real-time  systems  can  generate  three-dimensional  data  at  periods  on 
the  order  of  milliseconds  at  frame  rates  of  120Hz  and  higher  [18].  Target  data  can  be 
matched  to  a  library  of  object  marker  patterns  and  the  location  and  orientation  of  an 
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object  can  be  found.  Fast  processing  is  aided  by  using  retroreflective  markers  and  by 
calculation  of  the  interior  and  exterior  camera  parameters  prior  to  image  collection.  Ad¬ 
vances  in  subpixel  marking  techniques  including  the  use  of  greyscale  data  for  calculating 
the  centroid  of  targets  has  greatly  improved  the  accuracy  of  these  systems. 

2.1.5  Applications  of  photogrammetry  to  membrane  structures  measurement. 
Photogrammetry  has  several  characteristics  that  make  it  attractive  as  a  noncontact  mea¬ 
surement  tool: 

1.  It  is  relatively  inexpensive,  and  satisfactory  results  can  be  obtained  with  off-the- 
shelf  hardware. 

2.  It  is  adaptable  to  a  wide  variety  of  environments. 

3.  It  has  typical  accuracies  on  the  order  of  1  /10th  of  a  pixel 

4.  Data  are  captured  at  all  points  simultaneously. 

5.  Image  data  can  be  stored  for  reference  and  (additional)  processing  at  any  time. 

For  these  reasons,  it  is  being  adopted  in  a  wide  variety  of  fields.  Recent  research  has 
gone  into  developing  benchmarks  and  best  practices  for  photogrammetric  measurement 
of  specific  kinds  of  surfaces  [6,  13,  19].  In  fields  such  as  MAV  aerodynamic  research, 
photogrammetry  has  become  the  standard  measurement  technique  for  structures  such  as 
flapping  wings  [20-22], 

The  use  of  photogrammetry  to  measure  large  space  structures  has  been  steadily 
developing  over  the  past  ten  years.  Pappa  et  al.  [23]  described  in  a  2001  paper  the 
use  commercially  available  cameras  and  software  to  derive  the  surface  profile  of  a  5  m 
inflatable  antenna.  Retroreflective  targets  were  used  along  with  flash  photography  to 
create  defined  targets  on  the  structure,  and  results  showed  an  accuracy  on  the  order 
of  1:10,000.  Blandino  et  al.  [13]  investigated  the  use  of  videogrammetry  for  modal 
identification  in  membrane  structures  and  compared  the  results  with  those  from  scanning 
laser  vibrometer.  Pappa  et  al.  [14]  investigated  laser  dot  projection  of  targets  onto 
the  membrane  structure  to  maintain  high  target  contrast  without  the  use  of  physically 
attached  retroreflective  targets,  and  Dorrington  et  al.  [1]  used  fluorescent  coatings  to 
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generate  targets  on  transparent  membranes.  More  recently,  videogrammetry  has  been 
used  to  measure  in-plane  motion  of  a  thin  him  [15]. 

Photogrammetry  has  also  been  used  extensively  as  a  measurement  tool  in  MAV 
research.  Lightweight  and  flexible  materials  are  often  critical  design  components,  and  it 
would  be  difficult  or  impossible  to  attach  physical  sensors  without  affecting  their  dynamic 
response.  Pitcher  et  al.  [16]  gives  an  analysis  of  the  mode  shapes  of  the  wing  of  a 
Nighthawk  mini-UAV  at  various  angles  of  attack.  Three  synchronized  cameras  captured 
images  at  75  frames  per  second  of  the  Nighthawk  in  the  wind  tunnel.  The  wing  of  the 
nighthawk  was  covered  with  a  combination  of  coded  and  uncoded  targets  and  additional 
targets  were  placed  on  the  floor  around  the  vehicle  for  reference.  Results  showed  that 
the  videogrammetry  system  was  well-suited  to  measuring  the  large  displacements  of  the 
wing,  but  the  limited  resolution  of  the  system,  approximately  .5mm  per  pixel,  meant 
low-amplitude  wing  vibration  was  lost  in  noise.  However,  at  test  conditions  that  did 
produce  large  enough  displacements,  power  spectral  density  curves  were  generated  and 
animation  of  the  vibration  of  the  wing  at  resonant  frequencies  was  accomplished. 

Photogrammetry  techniques  have  become  the  standard  for  wing  shape  measure¬ 
ment  of  flapping  wings.  Curtis  et  al.  [20]  analyze  the  shape  of  a  number  of  wings  of 
different  shapes  and  materials  by  taking  synchronous  images  of  a  laser  dot  pattern  pro¬ 
jected  onto  the  surface  of  the  wing  while  it  flaps  on  a  test  stand.  Additional  printed 
dots  were  fixed  to  the  flapping  mechanism  for  reference.  The  images  are  processed  using 
PhotoModelcr1  photogrammetry  software,  and  a  time  history  of  the  surface  shape  was 
generated.  This  method  had  the  advantage  of  minimal  interference  with  the  structure 
of  the  wing  being  tested,  since  no  physical  targets  were  attached  to  the  wing  surface.  A 
denser  profile  can  be  obtained  with  a  digital  image  correlation  (D1C)  method,  a  texture- 
based  photogrammetry  technique,  as  described  in  Aono  et  al.  [21]  and  Chakravarty  and 
Albertani  [22],  A  random  speckle  pattern  is  applied  to  the  test  article  and  is  correlated 
between  synchronous  images.  Aono  et  al.  used  stereo  high  speed  cameras  to  capture 
images  of  a  flapping  wing  in  a  vacuum  chamber.  Chakravarty  and  Albertani  examined 
the  membrane  alone,  mounted  on  a  wire  ring,  undergoing  static  and  dynamic  deforma- 
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tions.  Both  systems  perform  temporal  tracking,  which  correlates  the  speckle  pattern 
across  epochs. 

All  techniques  used  in  MAV  wing  shape  and  space  structure  measurement  cur¬ 
rently  require  the  test  article  to  be  on  a  stand  or  actuator  and  the  cameras  and  other 
photogrammetry  equipment  placed  around  it.  This  tends  to  restrict  the  kind  of  testing 
that  can  be  performed  on  the  article  to  cruise  and  hovering  flight  conditions  in  the  case  of 
MAVs  and  fixed-average-position  tests  for  both  MAVs  and  membranes.  Measurement  of 
wing  shape  during  maneuvering  flight  has  not  been  found  in  the  literature  by  this  author, 
nor  has  deploying  or  translating  membranes.  Expanding  the  use  of  photogrammetry  to 
include  such  test  conditions  will  provide  unique  data  sets  for  improved  understanding  of 
the  characteristics  of  these  structures. 

Real-time  videogrammetry  is  widely  used  to  rapidly  prototype  automatic  control 
strategies.  By  moving  location  sensing  off-board  the  vehicle,  complex  control  strate¬ 
gies  can  be  tested  on  inexpensive  vehicles,  thereby  eliminating  the  costs  of  a  lightweight 
implementation  on  the  vehicle  and  the  risk  of  damage  in  the  case  of  a  vehicle  crash. 
Massachusetts  Institute  of  Technology’s  Real-time  indoor  Autonomous  Vehicle  test  En¬ 
vironment  (RAVEN)  [24]  and  Boeing’s  UAV  Swarm  testbed  [25-27]  host  tests  ranging 
from  cooperative  control  of  vehicles  to  nonlinear  control  for  aircraft  aerobatic  maneu¬ 
vers.  These  systems  require  many  cameras  for  maximum  redundancy.  These  systems 
are  popular  for  post-processing  applications  as  well,  in  the  entertainment  industry,  in 
biometric  analysis,  and  inverse  kinematics  [28] .  Though  other  efforts  have  been  made  at 
close-range  object  location  and  orientation,  real-time  videogrammetry  systems  are  the 
most  widely  accepted.  [29-31] 

2.2  Comparison  with  Other  Noncontact  Measurement  Systems 

Comparison  of  photogrammetry  to  other  noncontact  measurement  systems  helps  to 
illustrate  the  strengths  and  weaknesses  of  the  technique.  Laser  vibrometry  has  been  an 
accepted  measurement  tool  for  many  years,  and  is  often  used  as  a  control  measurement 
in  investigations  of  new  measurement  techniques  [13].  3-D  time-of- flight  laser  scanners 
share  similar  properties  to  laser  vibrometers,  but  measure  static  three  dimensional  po- 
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sition  rather  than  velocity.  Other  measurement  systems  such  as  Moire  interferometry, 
geometric  Moire,  and  capacitance  measurement  have  been  used,  especially  for  the  mea¬ 
surement  of  membrane  structures. 

Laser  vibrometry  is  the  measurement  of  velocity  of  an  object  based  the  interference 
of  a  laser  beam  with  the  reflected  response  from  an  object.  Based  on  the  phase  shift 
in  the  light  due  to  the  Doppler  effect,  relative  velocity  of  the  object  can  be  calculated 
directly.  Laser  vibrometry  is  a  point-based  measurement  device  which  takes  data  at 
specific  locations  on  an  object.  An  entire  surface  can  be  analyzed  by  collecting  data 
over  a  grid  of  points.  Data  can  be  recorded  at  frequencies  of  a  MHz  and  higher  and 
with  micrometer  resolutions.  As  noted  above,  videogrammetry  measurements  have  data 
collection  rates  limited  by  the  frame  rates  of  the  cameras  used,  and  systems  exceeding 
several  hundred  fps  are  rare. 

The  high  measurement  frequency  and  and  resolution  of  laser  vibrometers  come 
at  a  cost.  As  is  noted  in  [13],  laser  vibrometry  is  a  relatively  expensive  technology 
compared  to  videogrammetry,  with  single-point  vibrometry  systems  starting  at  $20,000 
and  scanning  systems  costing  over  $100,000.  The  performance  benefit  does  not  always 
justify  the  additional  cost  in  applications  such  as  large  space  structures,  where  vibration 
frequencies  are  often  quite  low.  Also,  Pitcher  et  al.  [16]  notes  that  large  amplitude 
vibrations  are  often  well  above  the  maximum  threshold  of  laser  measurement  systems. 
In  photogrammetric  systems,  the  amplitude  of  the  measurements  is  limited  by  the  depth 
of  field  of  the  cameras  being  used,  and  in  most  cases,  this  is  quite  large  in  comparison 
to  laser  vibrometers. 

3-D  time-of-flight  laser  scanning  measures  three  dimensional  locations,  and  shares 
some  of  the  characteristics  of  laser  vibrometers  in  that  it  is  not  affected  by  ambient 
lighting,  and  records  data  point  by  point.  Markley  et  al.  [2]  describe  this  measurement 
technique  as  it  relates  to  modeling  of  as-built  facilities  and  note  that  in  practice  its 
accuracy  is  comparable  to  photogrammetry,  but  that  it  is  much  less  time  consuming  than 
photogrammetry  to  initially  gather  data  when  analyzing  large  structures.  Generating 
models  from  the  data  from  either  measurement  technique  can  be  labor  intensive  and 
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prone  to  operator  error.  A  hybrid  technique  was  suggested  as  a  way  of  overcoming  the 
limitations  and  providing  redundancy. 

Moire  interferometry  has  the  benefit  of  being  a  full  field  measurement  technique, 
but  requires  a  diffraction  grating  or  grid  to  be  attached  to  the  test  object.  Other  in¬ 
terferometry  techniques  do  no  require  surface  preparation,  but  all  need  a  generally  flat 
surface  and  relatively  low  displacement  [32],  Capacitance  measurement  is  also  found  in 
the  literature  but  is  not  common  and  requires  a  flat,  conductive  surface  [33]. 

In  summary,  photogrammetry  is  the  preferred  option  in  situations  that  require 
versatility  and  a  moderate  degree  of  accuracy.  Its  full-held  capability  as  well  as  its 
ability  to  self-validate  its  data  are  useful  properties  in  many  applications.  As  imaging 
technology  improves,  photogrammetry  will  continue  to  get  more  accurate  and  reliable. 

2.3  Hybrid  Measurement  Systems 

The  term  “hybrid  measurement  system”  refers  to  the  combination  of  two  or  more 
measurement  systems  to  accomplish  a  single  task.  There  are  two  senses  in  which  the 
systems  are  combined:  data  merging  and  system  slaving.  Data  merging,  which  is  far 
more  common  in  the  literature,  is  the  combining  of  data  gathered  from  two  different 
measurement  systems  in  order  to  provide  redundancy  and  to  compensate  for  deficiencies 
in  the  individual  data  sets  [2,  34,  35].  System  slaving  is  the  controlling  of  one  local 
measurement  system  with  another  global  measurement  system.  This  method  is  useful 
when  there  is  one  particular  area  of  interest  that  could  be  at  an  arbitrary  location  in 
large  volume. 

The  concept  of  master-slave  system  relationship  for  sensors  is  used  in  the  area 
of  surveillance  camera  networks,  in  which  a  wide-angle  camera  directs  a  pan-tilt-zoom 
camera  to  a  location  of  interest  such  as  a  person  [36,  37].  The  purpose  of  these  systems  is 
to  provide  clear  images  that  allow  a  person’s  biometric  characteristics  to  be  identified  by 
an  operator.  Strictly  speaking  these  systems  should  be  distinguished  from  measurement 
systems,  since  precise  quantitative  data  is  not  required  on  the  object  of  interest. 
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Slaving  of  measurement  systems  is  a  new  area  of  study  begun  three  years  ago  at 
the  Air  Force  Institute  of  Technology.  A  hybrid  system  was  developed  which  involved 
a  real-time  videogrammetry  system  for  detecting  the  global  location  and  orientation  of 
an  object,  and  a  mirror  system  to  direct  a  laser  vibrometer  beam  onto  a  point  on  the 
object  [38,  39].  The  static  and  dynamic  error  of  the  aiming  system  was  determined 
and  vibrational  data  of  a  moving  object  was  shown  to  correlate  with  a  static  object 
case.  The  major  deficiency  of  the  system  was  the  time  delay  involved  in  calculating  the 
coordinates  of  the  object  by  the  videogrammetry  system  and  transmitting  the  data  to  the 
laser  steering  system.  The  delay  of  approximately  200  ms  caused  the  laser  to  point  at  a 
previous  location  of  the  object  and  therefore  limited  the  speed  at  which  the  object  could 
be  moved  and  still  have  the  laser  illuminate  it.  The  tracking  error  was  improved  with  the 
addition  of  a  Kalman  filter  to  the  tracking  algorithm,  but  the  fact  that  laser  vibrometry 
is  a  point  measurement  technique  still  presented  difficulties.  The  use  of  photogrammetry 
as  the  local  measurement  system  will  help  to  resolve  this  difficulty,  since  it  is  a  full-field 
measurement  technique. 

This  research  combines  a  real-time  videogrammetry  system  for  detecting  global 
location  of  the  test  object  and  four  PTZ  cameras  for  taking  high-speed,  high-resolution 
images  suitable  for  photogrammetry.  In  the  next  section,  the  details  of  the  system  design 
will  be  presented,  including  system  configuration,  the  determination  of  the  gimbal  and 
lens  settings,  and  the  software  controller. 
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III.  System  Design  and  Development 

3.1  System  Overview 

The  dynamic  tracking  and  measurement  (DTM)  system  consists  of  two  hybridized 
subsystems: 


1.  A  videogrammetry  system  to  capture  large  scale  3D  motion. 

2.  A  set  of  synchronized  pan-tilt-zoom  (PTZ)  cameras  that  can  capture  motion  at 
high  frame  rates. 

The  videogrammetry  system  calculates  the  position  of  the  object  in  real  time  using 
the  retroreflective  markers  placed  on  the  object.  The  object  coordinates  are  passed  to 
software  controllers  which  calculate  gimbal  azimuth  and  elevation  and  lens  zoom  and 
focus  for  each  PTZ  camera.  Images  are  captured  synchronously  at  up  to  500  fps  and 
stored  on  the  controller  computers.  Testing  equipment  includes  a  rotating  table  used  to 
constrain  dynamic  motion.  Each  component  is  coordinated  as  needed  through  hardware 
and  software  interfaces.  A  block  diagram  of  the  hybrid  system  is  shown  in  Figure  3.1, 
the  videogrammetry  camera  setup  is  shown  in  Figure  3.2.  A  PTZ  camera  is  shown  in 
Figure  3.3  with  the  rotating  table  in  Figure  3.4. 
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Figure  3.1:  The  DTM  system  consists  of  a  videogrammetry  subsystem  for  real-time 

motion  capture  and  four  pan-tilt-zoom  camera  subsystems. 
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Figure  3.2:  Videogrammetry  cameras  are  setup  around  the  room’s  perimeter  and  track 
body  motion.  Each  tracking  subsystem  uses  this  data  to  focus  at  the  moving  point. 


Figure  3.3:  A  camera  subsystem  consists  of  a  two  axis  gimbal,  zoom-focus  lens,  a  high 
speed  camera,  and  controller  computer  (not  shown). 


3.2  Videogrammetry  System 

Three  dimensional  position  measurement  of  an  object  of  interest  is  accomplished 
by  a  videogrammetry  system  built  by  Vicon1.  Ten  cameras  are  mounted  around  the  lab 
perimeter2  (see  Figure  3.2).  Retro-reflective  spherical  videogrammetry  targets  imaged  by 
cameras  are  marked  to  subpixel  accuracy.  The  2-D  locations  of  marked  targets  on  each 
image  are  then  sent  to  the  central  data  station  which  interprets  them  as  rays  through 
space.  Knowing  each  camera’s  location  and  orientation,  the  global  location  of  the  markers 
is  determined  by  ray  intersection  to  determine  a  3-D  location  in  the  videogrammetry 
coordinate  system  (VCS).  Sets  of  3-D  global  target  positions  are  compared  to  a  library 

1Vicon,  5419  McConnell  Avenue,  Los  Angeles,  CA  90066,  USA  moveme@vicon.com 

2The  videogrammetry  cameras  are  MX  T160  type  with  16  megapixel  resolution.  Software  used  was 
Tracker  vl.l.  The  working  capture  volume  is  approximately  5m  x  7m  x  2.5m.  Markers  of  3mm-8mm 
diameter  were  used. 
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(a)  Turntable  showing  mounted  (b)  Turntable  detail 

videogrammetry  object 

Figure  3.4:  Samples  can  be  mounted  on  a  platform  that  maintains  orientation  while 

moving  through  a  circular  path. 

of  objects.  For  each  pattern  match,  the  location  and  orientation  of  its  body  coordinate 
system  is  determined  with  respect  to  the  global  coordinate  system. 

The  videogrammetry  system  must  be  calibrated  prior  to  use,  after  re-orienting  the 
cameras,  and  after  normal  disturbances  have  affected  the  camera  positions  sufficiently. 
The  system  is  calibrated  by  using  a  specially  designed  videogrammetry  object  called  a 
calibration  wand,  shown  in  Figure  3.5.  The  wand  is  moved  about  the  volume  and  a 
number  of  frames,  typically  2500,  are  recorded  by  each  camera.  The  frames  are  pro¬ 
cessed,  and  the  root-mean-square  (RMS)  error  for  individual  camera  rays  are  calculated 
to  provide  insight  on  the  quality  of  the  calibration.  Typical  RMS  error  values  range  from 
.25  mm  to  .35  mm.  The  origin  of  the  VCS  is  set  by  placing  the  calibration  wand  at  the 
desired  location  and  recording  it  in  the  software. 

The  noise  of  the  position  measurement  was  characterized  by  the  perceived  motion 
of  a  stationary  object.  Translation  on  the  order  of  0.75  mm  and  rotation  on  the  order 
of  0.005°  were  observed;  both  well  below  the  requirements  for  centering  an  object  in  the 
held  of  view.  Similar  results  were  obtained  in  Ref.  [18].  The  videogrammetry  system 
has  an  update  rate  of  120  Hz  at  full  held  resolution. 

Objects  are  dehned  in  Vicon  by  selecting  a  collection  of  retrorehective  markers 
affixed  to  the  object  and  assigning  them  a  unique  identifier.  The  markers  must  be 
placed  sufficiently  nonsymmentrically  to  allow  the  software  to  determine  orientation, 
and  the  pattern  must  distinguishable  from  other  dehned  objects.  Spherical  markers  are 
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Figure  3.5:  Calibration  wand  used  to  calibrate  and  set  the  origin  of  the  videogramme- 
try  system 

the  most  robust,  since  they  appear  identical  from  any  direction,  but  two-dimensional 
circles  of  retroreflective  tape  can  also  be  used,  provided  that  the  viewing  angle  of  the 
videogrammetry  cameras  is  not  too  shallow. 

3. 3  Camera  Gimbals 

Each  PTZ  camera  is  steered  by  a  gimbal  (shown  in  Figure  3.3)  which  rotates  about 
perpendicular  axes.  The  bottom  servo  rotates  for  panning  (also  known  as  azimuth)  and 
the  top  servo  rotates  for  changing  elevation.  Elevation  and  azimuth  command  messages 
are  sent  by  serial  communication  to  the  gimbal  which  interprets  and  returns  its  current 
position.  The  nominal  position  accuracy  is  0.01°  and  maximum  slew  rate  is  120°  per 
second. 

3.3.1  Camera  aiming.  The  gimbal  control  software  calculates  the  azimuth 
and  elevation  angles  for  accurately  aiming  the  camera.  Assume  that  the  location  and 
orientation  of  the  gimbal  system  are  known  and  given  by  a  position  p,  and  unit  vectors 
for  the  body  x,  y  and  z  axes,  ~ct ,  b  and  respectively  and  that  the  location  of  interest 
is  "iif  with  all  vectors  given  in  the  VCS.  The  point  ~jJ  lies  at  the  intersection  of  the 
azimuth  rotation  axis  and  the  elevation  rotation  axis.  The  gimbal  body  coordinate 
system  (BCS)  0  axis  is  collinear  with  the  azimuth  axis  and  the  body  y  axis  is  collinear 
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with  the  elevation  rotation  axis  at  home  position.  The  directions  are  chosen  so  that  the 
system  is  right  handed  and  x  points  out  parallel  to  the  camera’s  optical  axis.  This  axes 
definition  ensures  that  tracking  angles  correspond  to  counterclockwise  rotations  about 
the  respective  axes.  Figure  3.6  illustrates  the  coordinate  systems  and  their  relationships. 


Figure  3.6:  Vector  f  is  the  distance  between  the  gimbal  and  test  object,  and  is  trans¬ 
formed  into  gimbal  body  coordinate  system  to  calculate  azimuth  and  elevation  angles. 

The  vector  V’  =  q  —  p  represents  the  line  connecting  the  camera  to  the  point 
of  interest.  The  transform  (” ,  b  ,  "F’])T  =  gives  the  line  in  BCS.  The  elevation 
angle,  -0,  is  found  by 

0  =arctan2  (3-1) 

where  arctan2  is  the  inverse  tangent  that  uses  the  signs  of  the  numerator  and  denominator 
to  distinguish  quadrant.  The  azimuth  angle,  0,  is  found  by 

0  =arctan2  (— .  (3.2) 

3.3.2  Camera  location.  In  order  to  calculate  azimuth  and  elevation  angles,  the 
gimbal  BCS  must  be  known  in  relation  to  the  VCS.  The  following  section  describes  several 
methods  of  determining  the  BCS.  The  BCS  can  be  located  by  hand,  by  photogrammetry, 
or  by  being  defined  and  located  by  the  videogrammetry  system.  Each  method  has 
advantages  and  disadvantages  which  will  be  discussed. 
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Hand  location  is  any  method  that  locates  the  BCS  by  manually  adjusting  the 
location  and  orientation  parameters.  This  can  be  as  simple  as  using  a  tape  measure  to 
find  the  x,  y  and  z  coordinates  and  aligning  the  BCS  and  VCS  orientation.  However, 
another  hand  method  was  developed  that  isolates  the  coordinates  and  allows  them  to  be 
adjusted  individually,  and  is  described  next. 

The  three  orientation  parameters  which  define  the  BCS  orientation  can  be  reduced 
to  one  z  axis  rotation  by  ensuring  the  gimbal  is  mounted  on  a  flat  surface  or  level  tripod  as 
shown  previously  in  Figure  3.3,  so  that  the  z  gimbal  axis  is  aligned  with  the  VCS  z  axis. 
A  step-by-step  method  was  developed  for  determining  the  three  location  coordinates  and 
z  axis  rotation  one  at  a  time.  The  rotation  angle  about  the  z  axis,  9 ,  is  found  by  tasking 
the  camera  to  point  along  the  BCS  x  or  y  axis.  The  VCS  x  and  y  axis  are  easily  visible 
on  the  lab  floor,  and  6  is  adjusted  until  the  camera  is  observed  to  be  pointing  along  the 
corresponding  VCS  axis.  The  z  coordinate  is  found  by  holding  a  videogrammetry  object 
at  the  same  height  as  the  camera,  and  reading  the  z  coordinate  of  the  object.  The  x 
and  y  coordinates  are  found  in  a  two  step  process,  illustrated  in  Figure  3.7.  First,  a 
videogrammetry  object  is  placed  along  either  the  x  or  y  axis,  and  the  tracking  system  is 
tasked  to  point  at  where  it  thinks  the  object  is  based  on  the  current  estimated  camera 
location.  The  other  location  parameter  is  adjusted  until  the  camera  is  pointing  at  the 
object.  Second,  the  object  is  moved  off  the  axis  and  the  final  parameter  is  adjusted  until 
the  camera  points  at  the  object. 


actual 

location 


a  estimated 
location 


(a)  y  coordinate  location 


(b)  x  coordinate  location 


Figure  3.7:  Videogrammetry  object  is  aligned  with  x  axis  and  y  camera  coordinate  is 
adjusted,  then  the  object  is  moved  off-axis  and  the  x  coordinated  is  adjusted. 
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The  method  described  above  produces  camera  locations  that  are  adequately  accu¬ 
rate  for  tracking  objects  in  motion  in  the  lab.  The  BCS  location  can  be  found  to  within 
approximately  2.5  cm  in  each  axis  direction.  Finer  adjustments  than  this  produce  no 
gain  in  aiming  accuracy  over  the  capture  volume,  due  to  errors  in  the  z  axis  orientation 
and  camera  mounting.  There  are  several  drawbacks  to  this  method.  The  method  must 
be  repeated  for  each  camera  location,  which  is  time-consuming.  Furthermore,  the  pos¬ 
sible  location  of  the  cameras  are  restricted  by  the  capture  volume.  The  camera  height 
cannot  be  measured  if  it  is  mounted  higher  than  the  maximum  height  the  videogramme- 
try  system  can  detect  an  object.  Therefore,  a  more  versatile  method  has  been  developed 
that  would  overcome  some  of  these  problems. 

The  new  method  uses  the  videogrammetry  system  and  PhotoModcler  photogram- 
metry  software  to  find  the  locations  and  orientations  of  all  cameras  at  the  same  time. 
This  method  requires  knowing  the  internal  calibration  parameters  for  each  camera  in  or¬ 
der  to  perform  accurate  photogrammetric  processing.  The  photogrammetry  processing 
generates  the  location  and  orientation  of  the  cameras  with  respect  to  an  arbitrary  coordi¬ 
nate  system  when  it  performs  the  bundle  adjustment  on  a  set  of  images.  By  transforming 
these  coordinates  into  the  VCS,  the  proper  camera  parameters  can  be  found. 

To  help  with  this  process,  a  special  targeted  object  is  used,  shown  in  Figure 
3.8.  This  targeted  object  has  retroreflective  videogrammetry  markers  and  PhotoModelcr 
coded  targets  on  it.  The  purpose  of  the  object  is  to  allow  the  videogrammetry  object 
coordinate  system  (VOCS)  and  the  photogrammetry  coordinate  system  (PCS)  to  be 
defined  identically.  This  reduces  the  number  of  coordinate  transformations  and  reduces 
error.  The  axes  and  scale  attached  to  the  targeted  object  are  defined  with  a  set  of  images 
of  the  object  before  taking  images  for  camera  location.  Any  calibrated  camera  can  be 
used  for  setting  up  the  targeted  object. 

The  videogrammetry  object  is  defined  such  that  the  origin  of  the  VOCS  is  located 
at  one  of  the  retroreflective  markers,  the  x  axis  aligned  with  another  marker,  and  the  y 
axis  in  the  plane  of  a  third.  The  coded  targets  are  used  to  speed  up  image  processing  by 
allowing  automatic  marking  and  referencing.  A  set  of  images  are  taken  with  a  calibrated 
camera,  and  a  flash  is  used  to  cause  the  retroreflective  markers  to  show  up  brightly.  The 
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coded  targets  are  automatically  marked  and  referenced  by  the  photogrammetry  software, 
and  the  high-contrast  retroreflective  markers  are  marked  and  referenced  by  hand  using 
the  software’s  subpixel  marking  tool.  The  images  are  processed  to  generate  the  three- 
dimensional  coordinates  of  each  referenced  point.  Since  the  retroreflective  markers  were 
referenced,  these  points  can  be  used  to  define  the  PCS  the  same  way  as  the  VOCS  was 
defined. 


Figure  3.8:  Targeted  object  used  for  camera  location.  Retroreflective  and  coded  targets 
are  shown.  The  VOCS  and  PCS  x  and  y  axes  are  shown,  with  the  origin  at  their 
intersection  and  the  z  axis  oriented  so  as  to  form  a  right  hand  coordinate  system. 

The  photogrammetry  project  and  the  images  used  to  set  up  the  targeted  object 
are  duplicated  and  reused  for  every  new  camera  location  project.  The  targeted  object 
will  only  need  to  be  redefined  if  either  the  retroreflective  markers  or  coded  targets  are 
changed  or  disturbed.  Therefore  it  is  important  that  they  be  securely  attached  to  the 
object. 

Once  the  targeted  object  is  defined,  it  can  be  used  to  locate  the  PTZ  cameras. 
This  can  be  done  one  at  a  time  or  in  groups,  the  limiting  factor  being  the  ability  of 
each  camera  to  see  the  coded  targets  on  the  targeted  object.  An  additional  restriction 
is  that  the  internal  parameters  of  each  of  the  PTZ  cameras  must  be  known.  These  can 
be  determined  from  a  calibration  at  the  current  camera  state,  or  by  approximating  the 
parameters  using  curve  fits  to  a  set  of  calibrations.  Curve  fit  calibration  parameters  will 
be  described  in  a  later  section.  The  most  important  parameter  for  camera  location  is 
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the  focal  length.  It  was  found  to  be  difficult  to  get  the  required  accuracy  for  camera 
location  with  a  curve  fit  approximation,  and  the  best  results  have  been  obtained  by  using 
a  camera  state  at  which  the  camera  calibration  routine  has  been  run. 

The  targeted  object  is  placed  in  a  location  where  each  camera  can  see  it.  The 
location  and  orientation  of  the  targeted  object  in  the  VCS  is  recorded,  and  images  of  the 
targeted  object  are  taken  with  each  camera.  If  desired,  the  targeted  object  can  be  rotated 
and  additional  sets  of  images,  locations  and  orientations  can  be  gathered  to  provide 
redundant  locations  for  error  checking.  The  images  are  imported  into  the  PhotoModeler 
project  set  up  previously,  and  they  are  automatically  marked  and  processed  using  the 
internal  camera  parameters.  The  processing  generates  camera  locations  and  orientations 
in  the  PCS. 

The  final  step  is  to  transform  the  PCS  locations  and  orientations  into  VCS.  The 
PCS  locations,  [xp,  yp,  %,]T,  are  transformed  by  multiplying  by  a  rotation  matrix  derived 
from  the  orientation  of  the  targeted  object  in  VCS,  defined  by  body  1-2-3  angles  (f>v, 
9V,  and  and  adding  it  to  the  translation  vector  from  the  VCS  origin  to  the  targeted 
object,  [xv,yv,zv]T. 
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where  CpcStVCS  is  the  rotation  matrix  from  PCS  to  VCS  given  by 
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This  transformation  gives  the  location  of  the  camera  in  VCS,  [xc,  yc,  zc]T ,  which  can  be 
directly  entered  into  the  tracking  controller. 
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Since  we  have  assumed  that  the  PTZ  camera  z  axis  is  aligned  with  the  VCS  z 
axis,  there  is  only  one  camera  rotation  parameter  to  find,  the  rotation  about  the  z  axis, 
0.  The  camera  coordinate  system  (CCS)  in  PCS  is  given  in  body  1-2-3  form,  with  the 
camera  x  axis  pointing  to  the  right  of  the  image  plane  and  y  axis  pointing  up,  if  looking 
at  the  camera  from  behind  (illustrated  previously  in  Figure  2.1).  The  z  axis  lies  along 
the  optical  axis  pointing  the  opposite  direction  of  the  camera.  Thus  the  negative  z 
axis  points  out  the  front  of  the  lens.  A  rotation  matrix  Cccs  pcs  can  be  formed  from  the 
rotation  angles  (j>p,  6p,  and  and  multiplying  this  by  the  rotation  matrix  from  PCS  to 
VCS  gives  the  camera  orientation  in  VCS.  The  unit  vector  passing  through  the  front  of 
the  lens  is  given  by 
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The  angle  6  is  found  by  subtracting  the  angle  between  the  projection  of  the  vector  out 
the  front  of  the  lens  onto  the  VCS  x-y  plane  and  the  VCS  x  axis  from  the  gimbal  azimuth 
angle.  First  evaluate  the  cross  product  of  the  projection  of  the  camera  ray  on  the  VCS 
x-y  plane  and  the  x  axis. 
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The  angle  between  the  two  vectors  is  given  by 
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Subtracting  O'  from  the  gimbal  azimuth  angle  at  which  the  photograph  was  taken  gives 
the  desired  rotation  value  6. 

These  transformations  are  performed  in  a  MATLAB  script  which  has  as  inputs  the 
VCS  and  PCS  data  and  outputs  the  camera  location  parameters  and  the  angle  O'  of  the 


camera.  O'  must  be  subtracted  from  the  gimbal  azimuth  angle  manually  to  determine 
the  6  parameter. 

To  test  the  effectiveness  of  this  method  of  locating  the  cameras,  twenty  camera 
locations  were  generated  based  on  three  separate  projects.  During  project  one  and  two, 
the  PTZ  camera  was  mounted  on  a  tripod  about  1.5  m  off  the  floor  and  about  3  m  from 
the  center  of  the  room  and  the  targeted  object  was  placed  on  the  floor  in  the  center  of 
the  room.  During  project  three,  another  camera  was  included  to  illustrate  the  process 
of  locating  more  than  one  camera  at  the  same  time.  The  VCS  origin  was  defined  to  be 
in  the  center  of  the  room  but  at  table  height,  approximately  0.6  m  above  the  floor. 

The  PTZ  camera  was  calibrated  at  the  lens  settings  that  were  used  for  image 
capture.  The  lens  settings  were  not  changed  between  projects.  Images  were  captured 
of  the  targeted  object  during  each  of  the  first  two  projects.  The  targeted  object  was 
rotated  between  each  image,  and  the  videogrammetry  coordinates  of  the  new  location 
was  recorded  each  time.  The  PTZ  camera  orientation  was  not  changed.  The  set  of  images 
was  imported  into  a  PhotoModeler  photogrammetry  software  project  which  included  the 
“set-up”  images  which  were  used  earlier  to  define  the  PCS.  The  photos  were  processed, 
and  the  locations  and  orientations  of  the  camera  in  the  PCS  were  exported.  This  data, 
along  with  the  location  of  the  targeted  object  in  each  image,  were  imported  into  the 
MATLAB  script  and  the  transforms  described  above  were  applied. 

Project  three  was  processed  in  an  identical  way,  except  an  additional  PTZ  camera 
took  images  at  each  of  four  targeted  object  positions.  In  order  for  the  object  to  be  in  view 
of  both  cameras,  the  object  was  moved  from  the  center  of  the  room  and  the  orientations, 
but  not  location,  of  the  PTZ  cameras  was  changed  from  project  one  and  two. 

The  data  from  each  project  for  the  primary  PTZ  camera  is  shown  in  Table  3.1. 
Listed  is  the  standard  deviation  in  each  direction,  and  the  distance  that  the  average  of 
all  generated  locations  were  from  the  hand  location  method.  Even  considering  the  uncer¬ 
tainty  of  the  hand  location  method,  approximately  2.5  cm  in  each  direction,  the  average 
value  is  consistently  outside  this.  This  makes  intuitive  sense,  since  the  photogrammetry 
method  of  location  measures  from  the  ideal  focal  point  of  the  lens,  which  does  not  coin¬ 
cide  with  the  origin  of  the  gimbal  axes.  In  other  words,  the  origins  of  the  CCS  and  the 
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gimbal  BCS  do  not  in  general  coincide.  This  fact  illustrates  the  principle  deficiency  of 
location  of  the  cameras  by  this  method.  There  is  currently  no  simple  way  to  relate  the 
focal  point  to  the  origin  of  the  gimbal  axes. 

Figure  3.9  shows  the  camera  locations  generated  by  the  photogrammetry  method 
and  the  hand-generated  gimbal  location.  It  can  be  seen  that  the  photogrammetry- 
generated  locations  are  generally  located  closer  to  the  object  than  the  hand-generated 
locations.  Also,  project  three  data  are  rotated  counterclockwise  around  the  control  point, 
corresponding  to  the  rotation  of  the  camera  for  that  project.  Thus  the  data  appear  to 
roughly  lie  on  the  optical  axis  of  the  camera  ahead  of  the  gimbal  origin  (see  Figure  3.10). 
It  is  supposed  that  with  additional  data,  a  correction  factor  could  be  derived  that  would 
account  for  the  error,  considering  that  the  data  are  generally  precise.  In  particular,  ad¬ 
ditional  data  should  be  gathered  on  the  dependence  of  the  generated  locations  on  the 
lens  settings.  Focal  length  and  focus  setting  may  have  a  significant  effect  on  the  position 
of  the  ideal  focal  point. 

It  was  found  that  the  photogrammetry  method  of  locating  the  cameras  was  precise 
but  less  accurate  than  the  hand  calibration  method.  Although  highly  accurate  location 
of  the  cameras  is  not  necessary,  deviations  over  5  cm  produce  noticeable  aiming  errors 
which  could  result  in  lost  image  data.  Potential  benefits  of  multi-camera  location  are 
currently  outweighed  by  the  need  to  calibrate  the  cameras  prior  to  use.  If  improved 
curve  fits  to  the  calibration  parameters  are  developed  which  would  obviate  the  need  for 
pre-calibration,  as  well  as  a  correction  factor  for  the  difference  between  focal  point  and 
gimbal  origin,  this  method  may  become  useful  for  large  multiplc-PTZ-camera  systems 
where  the  cameras  are  located  outside  the  capture  volume. 


Table  3.1:  Data  from  camera  location  by  photogrammetry 
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Figure  3.9:  Planar  graphs  of  camera  locations  with  respect  to  hand  location  point 

(denoted  by  circle).  Bine  x  denotes  project  one  and  two  points.  Red  square  denotes 
project  three  points,  found  after  rotating  camera.  Scale  is  in  millimeters. 

A  third  camera  location  technique  involves  using  retroreflective  videogrammetry 
targets  to  register  the  gimbals  as  videogrammetry  objects,  with  their  origin  at  the  gimbal 
axes  intersection.  This  is  the  most  direct  approach  to  camera  location;  since  the  goal 
is  to  know  the  camera  locations  in  the  VCS,  it  makes  sense  to  have  them  located  by 
the  videogrammetry  system.  Furthermore,  once  the  gimbal  videogrammetry  objects  are 
defined,  there  is  very  little  additional  work  for  the  operator.  The  camera  stands  can 
be  moved  to  a  new  location  and  the  videogrammetry  system  will  record  the  location 
and  pass  it  to  the  aiming  software  automatically.  However,  this  method  requires  that 
the  cameras  be  placed  within  the  capture  volume  of  the  videogrammetry  system,  and 
therefore  reduces  the  amount  of  the  volume  that  can  be  used  for  test  articles.  This  can 
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Figure  3.10:  Relation  of  the  gimbal  rotation  axes  (red)  to  the  optical  axis  and  ap¬ 

proximate  focal  point  of  the  camera  (blue).  The  gimbal  x  axis  and  the  optical  axis  are 
parallel  (but  not  collinear)  when  the  gimbal  is  at  zero  azimuth  and  elevation. 

be  a  significant  problem,  since  typical  ten-camera  videogrammetry  capture  volume  is 
not  more  than  a  few  meters  on  each  side,  and  the  minimum  focal  distance  of  the  PTZ 
cameras  is  1.5  m.  This  location  method  is  preferred  for  its  ease  of  use,  but  it  is  not 
always  possible  to  implement. 

The  gimbal  objects  are  defined  by  placing  retroreflective  markers  on  the  bottom 
fixed  servo  and  creating  an  object  in  the  videogrammetry  software.  The  origin  of  the 
newly  defined  object  must  then  be  manually  translated  to  the  intersection  of  the  gimbal 
axes.  This  is  done  with  the  aid  of  temporary  markers  placed  on  the  camera  approximately 
on  the  axes  of  the  gimbal.  Finally,  the  gimbal  is  rotated  to  zero  azimuth  angle,  and  the 
x  axis  of  the  object  coordinate  system  is  rotated  to  align  with  the  camera’s  optical  axis. 
When  the  gimbal  is  placed  in  the  videogrammetry  capture  volume,  the  location  and 
orientation  is  generated  and  can  be  passed  to  the  aiming  software. 

Videogrammetry  markers  can  be  located  to  sub-millimeter  accuracy,  and  so  the 
accuracy  of  the  location  of  the  gimbal  origin  is  limited  only  by  the  ability  to  place  the 
temporary  markers  on  the  gimbal  axes  for  correct  alignment  of  the  object  coordinate 
system.  The  directions  of  the  gimbal  axes  are  well-defined  on  the  gimbal,  by  the  rotating 
shafts  extending  outside  the  case. 
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The  three  methods  described  above  provide  a  variety  of  ways  to  locate  the  PTZ 
cameras.  Camera  location  is  a  necessary  step  for  aiming,  zooming  and  focusing  the 
cameras  on  a  moving  object. 

3.4  Lens 

Photogrammetric  accuracy  is  primarily  dependent  upon  the  resolution  of  the  object 
in  images  used  for  processing.  Therefore  it  is  desirable  for  the  test  object  to  fill  as  much 
of  the  frame  as  possible.  Zoom  and  focus  lenses  are  used  to  maintain  a  sharp  image  and 
a  desired  viewing  radius  at  the  distance  of  the  object. 

Four  closed  circuit  television  lenses  are  used  to  control  aperture,  magnification, 
and  focus  of  the  pan-tilt-zoom  cameras.  Commands  are  sent  to  the  lens  via  16  bit  serial 
protocol.  The  lens  is  capable  of  viewing  angles  from  29°  to  2°,  and  can  focus  on  objects 
as  close  as  1.5  m.  The  lens  can  frame  and  focus  on  a  0.5  m  diameter  sphere  at  any 
distance  between  1.75  m  and  14  m,  although  typical  distances  in  the  current  lab  are 
between  1.75  m  and  4  m. 

The  magnification  and  focus  settings  for  each  lens  are  determined  from  the  distance 
of  the  object  and  the  desired  field  of  view  at  that  distance.  In  the  system,  the  lens  acts 
as  a  varifocal  lens,  in  that  the  magnification  and  the  focus  are  functions  of  one  another; 
however,  the  effect  of  changing  the  focus  on  the  magnification  was  minimal  across  all 
magnification  settings,  and  so  was  neglected. 

Functions  of  magnification  and  focus  were  determined  from  collected  data.  First, 
the  angle  of  view  was  recorded  for  seven  magnification  settings,  with  the  focus  setting 
at  its  minimum.  An  empirical  equation  was  developed  for  the  magnification  setting  as  a 
function  of  the  viewing  angle  using  a  least  squares  fit  of  a  polynomial  curve  to  interpolate 
between  the  points.  Letting  r  be  the  distance  to  the  object  and  d  be  the  width  of  the 
field  of  view  at  that  distance,  the  viewing  angle  u  is  given  by 

uj  —2  arctan(d/2r)  (3.8) 
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Note  that  a  maximum  u  of  19°  is  found  at  the  minimum  distance  of  1.5  m  and  largest 
typical  viewing  radius  of  .25  m  in  the  lab.  A  fifth  order  polynomial  of  the  form 

Zs  =cq  -\-  C\UJ  T  C20J~  A  c^co^  T  CiCU  -t-  c§oj^  (3.9) 

was  found  to  produce  good  magnification  results.  Figure  3.11(a)  shows  the  curve  fit  and 
the  data  points.  At  large  u  the  curve  fit  is  unrealistic,  but  for  values  of  u  below  19°  the 
curve  adjusts  the  field  of  view  with  sufficient  accuracy. 

The  focus  setting  function  was  derived  using  a  bivariate  interpolation  of  magnifi¬ 
cation  and  focus  data.  Magnification  and  focus  data  were  collected  at  six  distances,  and 
six  hyperbolic  polynomials  were  fit  to  these  data. The  choice  of  function  and  polynomial 
order  were  made  based  on  observation  of  general  trends  in  the  data.  Let  Fs  be  the  focus 
setting. 


Fs  —  +  ai(l/Zs)  +  a-2(l/Zs)2  (3.10) 

where  oo,  «i  and  a2  are  polynomial  coefficients.  Another  set  of  1st  order  polynomials 
were  fit  to  the  coefficients  as  a  function  of  distance,  shown  in  Figure  11(b). 

a*  =  bifi  +  biylr  *  =  0,1,2  (3.11) 

and  so 

Fs  =  S>,o  +  bit\d)  \  (3-12) 

i= 0  \  s/ 

Figure  3.12  shows  the  initial  data  and  the  distances  at  which  it  was  gathered  in  blue  and 
the  curve  fits  based  on  that  data  in  red.  Although  the  curve  fits  do  not  match  the  data 
very  well  in  some  regions,  especially  at  the  edges  of  the  capture  volume,  in  the  typical 
region  for  this  lab  the  curve  fits  are  reasonable.  The  green  hashed  line  shows  the  lower 
distance  limit  from  the  camera  and  two  lines  of  constant  field  of  view  (FOV)  at  which 
typical  test  articles  are  photographed. 
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(a)  Graph  of  lens  magnification  setting  vs  viewing  (b)  Graph  of  focus  polynomial  coefficients 
angle. 


Figure  3.11:  Graph  of  data  and  curve  fits  for  magnification  and  focus  polynomial 

coefficients 


Figure  3.12:  Graph  showing  the  magnification  and  focus  data  (blue),  the  curve  fit 

approximations  (red)  and  the  typical  domain  of  the  lens  settings  in  the  current  lab 
(green) . 
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Note  also  that  at  short  distances  the  focus  setting  appears  to  be  very  sensitive  to  the 
magnification  setting.  The  2.26  m  distance  curve  covers  the  entire  range  of  focus  settings 
over  only  approximately  15,000  units  of  magnification.  This  is  somewhat  misleading. 
While  it  is  true  that  optimal  focus  is  highly  dependent  on  magnification  on  these  curves, 
the  depth  of  focus  is  increased  at  lower  magnification  settings.  Therefore  there  is  a 
greater  tolerance  for  error  in  the  focus  setting,  and  conditions  that  according  to  this 
graph  are  less  than  ideal  nonetheless  produce  suitable  images. 

Another  feature  to  notice  in  Figure  3.12  is  trimming  of  part  of  the  domain  at  low 
FOV  settings  and  short  distances.  This  effect  must  be  taken  into  account  when  preparing 
the  camera  system  for  image  capture.  The  PTZ  cameras  may  need  to  be  placed  further 
outside  the  capture  volume  for  smaller  objects. 

Figure  3.13  illustrates  the  entire  process  of  calculating  the  lens  settings.  This 
method  of  generating  the  focus  and  magnification  settings  provides  good  clarity  and 
framing  of  an  object  within  the  capture  volume.  Combining  the  aiming,  zoom  and  focus 
systems  produces  the  PTZ  camera  system  which  provides  framing  and  focusing  from  four 
viewpoints. 

3. 5  Camera 

Each  camera  is  capable  of  taking  500  frames  per  second  at  1280x1024  pixel  res¬ 
olution.  Synchronous  image  capture  is  achieved  by  having  the  cameras  capture  frames 
based  on  an  external  trigger.  The  trigger  is  generated  by  a  hardware  timer  in  the  control 
computer  and  sent  to  each  of  the  cameras. 


Figure  3.13:  Schematic  describing  the  calculation  of  magnification  and  focus  settings 
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3.6  Computers  and  Software 

Each  of  the  four  cameras  that  comprise  the  PTZ  camera  system  has  a  dedicated 
computer  containing  control  software  and  image  storage.  The  control  software  is  used 
for  calculating  and  communicating  gimbal  and  lens  commands  as  described  above  and  to 
record  current  gimbal  and  lens  states.  Another  software  package  performs  image  capture 
and  records  the  images  to  a  2  terabyte  RAID3  array.  A  fifth  computer  is  used  to  generate 
the  trigger  signal  for  the  cameras,  and  it  can  also  be  used  to  control  each  of  the  four 
camera  computers  over  the  network  so  that  an  image  capture  can  be  set  up  and  run 
from  one  station.  The  3-D  position  of  a  test  object  is  obtained  from  the  videogrammetry 
system  software  run  on  a  dedicated  computer.  All  computers  are  connected  with  a  gigabit 
local  area  network.  Figure  3.14  illustrates  the  communication  between  the  software  and 
hardware. 

The  PTZ  controller  software  is  implemented  in  the  data-flow  programming  soft¬ 
ware  Lab  VIEW4.  The  controller  consists  of  four  main  parts:  Data  acquisition  from  the 
videogrammetry  software,  gimbal  controller,  lens  controller,  and  lens  and  gimbal  state 
recorder.  These  parts  are  placed  in  separate  programming  loops  for  optimal  speed  for 
each  part.  Figure  3.15  shows  the  control  software  interface. 

Data  acquisition  is  accomplished  using  a  .NET  library  supplied  by  Vicon.  The 
controller  software  uses  this  library  to  call  for  the  location  and  orientation  of  specific  ob¬ 
jects  from  the  videogrammetry  software.  If  the  camera  is  registered  as  a  videogrammetry 
object,  its  location  is  requested  at  the  beginning  of  tracking  also.  The  videogrammetry 
system  produces  data  at  120  Hz  at  full  16  megapixel  resolution. 

The  gimbal  controller  implements  the  aiming  algorithm  described  in  Sections  3.3 
and  3.4.  At  each  iteration  of  the  gimbal  loop,  the  azimuth  and  elevation  is  calculated  and 
compared  to  the  previous  command.  If  the  change  is  greater  than  a  certain  threshold, 
the  command  is  send  to  the  gimbal.  Azimuth  and  elevation  are  sent  at  one  time  and 
additional  commands  to  new  angles  override  previous  ones.  If  the  change  is  less  than  the 
threshold,  no  command  is  sent  to  the  gimbal  in  order  to  avoid  unnecessary  use  when  the 
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Figure  3.14:  System  software  and  hardware  communication 


Figure  3.15:  The  lens,  gimbal  and  lens  and  gimbal  state  recording  is  controlled  with 
the  interface  shown. 
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tracked  object  is  stationary.  Typical  thresholds  are  0.01°.  Whether  or  not  a  command 
is  sent,  the  controller  program  queries  the  current  azimuth  and  elevation  of  the  gimbal. 

The  lens  controller  implements  the  focus  and  zoom  algorithm  described  above.  The 
appropriate  zoom  and  focus  setting  is  calculated,  and  a  similar  thresholding  is  applied 
as  to  the  gimbal.  Typical  threshold  values  for  both  zoom  and  focus  are  1,000  units.  The 
commands  must  be  sent  in  a  series;  sending  a  focus  command  before  the  zoom  command 
is  completed  will  interrupt  the  zoom.  Waiting  for  previous  commands  to  complete  limits 
how  quickly  new  commands  can  be  sent.  After  each  command,  the  lens  state  is  queried 
repeatedly  as  the  setting  changes,  resulting  in  knowledge  of  the  zoom  and  focus  settings 
throughout  the  change. 

The  gimbal  and  lens  controller  loops  run  at  the  maximum  speed  allowed  by  each 
device.  The  gimbal  loop  averages  approximately  15  ms  for  one  iteration  when  no  move¬ 
ment  command  is  sent,  and  22  ms  when  commands  are  sent.  The  lens  loop  must  wait 
until  each  zoom  and  focus  command  is  completed  before  restarting  and  potentially  send¬ 
ing  a  new  command.  The  time  this  takes  varies  depending  on  how  large  a  change  is 
directed,  but  can  be  500  ms  or  more. 

The  object  location,  gimbal  state,  and  lens  state  are  all  saved  into  an  array  for 
use  in  photogrammetric  image  processing  in  a  data  recording  loop.  The  recording  loop 
is  a  timed  loop  that  runs  with  a  5  ms  period  in  order  not  to  miss  any  data  from  the 
other  loops.  Additionally,  the  recording  loop  period  and  loop  iteration  is  recorded  to 
alert  to  any  error  in  timing.  The  timing  of  this  loop  is  critical,  as  it  will  be  the  basis  of 
synchronizing  the  gimbal  and  lens  state  information  with  the  image  sequence. 

The  images  and  the  lens  and  gimbal  state  data  are  used  by  the  photogrammetry 
software  to  generate  surface  profiles.  The  state  data,  in  particular  the  zoom  setting  of 
the  lens,  is  used  to  determine  the  camera  calibration  parameters  as  is  described  in  the 
next  chapter.  With  the  calibration  parameters,  the  images  can  be  processed  and  3-D 
point  locations  found. 
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3. 7  Image  Processing 

Image  processing  is  performed  with  the  commercial  photogrammetry  software  Pho- 
toModeler  Scanner.  The  software  contains  subpixel  marking  tools,  coded  target  recog¬ 
nition,  and  texture-based  photogrammetry  tools. 

Although  common  image  processing  routines  are  well  developed  and  automated 
within  PhotoModcler,  dynamic  tracking  and  measurement  requires  additional  flexibility 
due  to  the  dynamic  internal  and  external  camera  parameters,  the  motion  of  the  test  ob¬ 
ject,  and  the  large  number  of  images  to  be  processed.  The  necessary  custom  automation 
can  be  brought  about  by  controlling  PhotoModeler  externally  using  dynamic  data  ex¬ 
change  and  a  scripting  software  such  as  MATLAB.  The  majority  of  the  photogrammetry 
processes,  with  the  notable  current  exception  of  texture-based  surface  mapping,  can  be 
controlled  in  this  way. 

3.8  Test  Equipment 

The  dynamic  accuracy  of  the  aiming  system  was  measured  using  a  rotation  table 
shown  previously  in  Figure  3.4.  The  rotation  table  allowed  infinite  duration  movement 
along  a  circular  path.  The  mounting  platform  at  the  end  of  the  arm  counterrotates 
with  the  turntable  itself  by  means  of  a  gear  and  belt  system.  This  keeps  the  mounted 
object  facing  the  same  direction  and  all  points  on  the  object  maintaining  the  same  speed 
throughout  the  rotation.  A  box  with  several  coded  targets  on  its  face  was  defined  in  the 
videogrammetry  software  and  was  tracked  for  the  dynamic  tests. 
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IV.  System  Tests  and  Results 


FOUR  primary  tests  were  performed  to  evaluate  the  capabilities  of  the  DTM  sys¬ 
tem,  and  their  methodology  and  results  are  presented  in  this  chapter.  First,  a 
description  of  the  use  of  curve  fits  for  determining  calibration  parameters  is  given  and 
an  analysis  of  suitability  of  this  approach  is  presented.  Image  synchronization  is  demon¬ 
strated.  The  camera  aiming  error  is  evaluated  the  the  limits  it  places  on  PTZ  tracking 
is  discussed.  Finally,  the  tracking  and  image  capture  of  a  MAV  in  flight  is  presented  and 
shown  to  correspond  to  the  predicted  camera  aiming  performance. 

4-1  Camera  Calibration 

Determining  the  3-D  location  of  a  point  based  on  its  two  dimensional  position  on 
the  photographic  plane  requires  an  accurate  camera  model.  Typically,  the  parameters  of 
the  model  are  solved  for  with  a  camera  calibration.  For  a  fixed-parameter  camera,  the 
parameters  can  be  solved  for  by  taking  8-12  images  of  a  calibration  grid  and  performing 
a  bundle  adjustment  using  photogrammetry  software  such  as  PhotoModeler.  However, 
as  stated  before,  many  of  the  camera  calibration  parameters  are  functions  of  the  zoom, 
focus,  and  aperture  of  the  lens,  and  so  a  new  camera  model  is  needed  for  every  com¬ 
bination  of  these  settings.  This  dependence  poses  a  significant  problem  to  the  designer 
of  a  photogrammetry  system  which  allows  the  zoom  and  focus  settings  to  change  freely 
throughout  a  sequence  of  images.  The  calibration  parameters  must  be  derived  as  a 
function  of  a  known  value,  such  as  the  zoom  and  focus  settings. 

4-2  Development  of  Parameter  Curves 

For  the  DTM  system,  empirical  functions  were  developed  for  the  camera  model 
parameters  based  the  calibration  data  from  many  lens  conditions.  This  method  for  cali¬ 
brating  the  pan-tilt-zoom  cameras  is  a  modified  version  of  the  approaches  in  References 
[10]  and  [12],  which  were  described  in  section  2.1.3.  In  both  papers,  the  camera  pa¬ 
rameters  which  were  determined  have  the  greatest  effect  on  the  photogrammetry  results 
and  to  be  the  easiest  to  model  were  the  principal  point  location  (xo,yo),  first  distortion 
parameter  k\ ,  and  focal  length  /. 
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The  camera  parameters  chosen  for  this  model  were  f,k\,  and  k2.  These  parameters 
were  chosen  primarily  for  the  ease  with  which  they  could  be  modeled.  Their  values  as 
a  function  of  zoom  setting  showed  clear  trends  which  enabled  reliable  curve  fitting. 
Numerous  calibrations  showed  the  values  of  decentering  distortion  parameters  p\  ad 
p2  had  little  effect  on  the  bundle  adjustment  solution  and  usually  were  automatically 
removed  by  the  calibration  algorithm.  Principal  point  parameters  were  held  constant  for 
simplicity  but  will  be  considered  for  inclusion  in  future  work. 

The  camera  model  is  based  on  the  model  presented  in  section  2.1.2  and  is  given  in 
Equation  4.1 


where 


1 

* 

1 

t-1 

1 

x  —  Xq  +  dx 

Y  -  Yl 

“  k 

y  -  Vo  +  dy 

z  ZL 

(4.1) 


dx  =  ki(x  —  x0)r2  +  k2(x  —  x0)r4 

(4.2) 

dy  =  kl(y-  y0}r2  +  k2(y  -  y0)r 4 

(4.3) 

The  parameters  /,  k\ ,  and  k2  were  calculated  using  PhotoModcler’s  self-calibration 
algorithm.  Twenty-two  zoom  and  focus  points  in  the  normal  operating  range  of  the  lens 
were  calibrated.  The  aperture  was  fixed  at  full  dilation  and  was  not  considered  for 
dependence.  The  data  showed  that  the  parameters  were  a  much  stronger  function  of  the 
zoom  setting  than  the  focus,  as  was  expected,  and  so  the  parameters  were  developed  as 
a  functions  of  zoom  only.  Curves  were  fit  to  the  data  based  on  a  weighted  least  squares 
approximation,  where  the  weight  for  each  data  point  is  the  inverse  square  of  the  RMS 
residual  value  produced  by  the  PhotoModeler  calibration.  These  curve  fits  are  shown  in 
Figure  4.1. 

These  curves  allow  for  efficient  estimation  of  the  calibration  parameters  from  the 
zoom  setting,  which  is  recorded  during  image  capture,  avoiding  a  lengthy  self-calibration 
at  each  time  step. 
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(a)  Focal  length  curve  fit  (b)  k\  parameter  curve  fit 


(c)  &2  parameter  curve  fit 


Figure  4.1:  Graphs  showing  calibration  parameter  data  and  curve  fits.  Error  bars  are 
based  on  calibration  quality  and  determine  weights  given  to  data  points 
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4-3  Test  of  Curve- fit  Parameters 

The  performance  of  the  photogrammetric  processing  with  the  use  of  curve-fit  pa¬ 
rameters  was  evaluated  by  comparing  the  resulting  point  locations  with  those  when  using 
parameters  generated  by  a  partial  and  full  calibration.  This  test  is  intended  to  give  some 
insight  into  the  error  incurred  by  using  a  less  sophisticated  camera  model,  and  then 
interpolating  the  camera  parameters  between  calibration  points. 

The  following  metrics  were  used  for  the  evaluation.  The  internal  consistency  of  a 
point  cloud  is  determined  by  the  RMS  residual  and  tightness  value.  The  residual  gives 
the  two-dimensional  distance  of  the  location  of  the  point  in  a  particular  image  from 
the  location  of  the  solution  point  generated  from  all  the  images.  The  tightness  value  is 
similar,  but  is  the  3-D  distance  from  the  particular  image  point  to  the  solution  point. 
Absolute  point  accuracy  was  measured  against  the  “truth”  point  cloud  generated  from 
fully  calibrated  parameters.  The  RMS  point  error  was  calculated  in  each  direction  as  well 
as  the  total  RMS  error.  Since  photogrammetric  accuracy  is  dependent  on  the  resolution 
of  the  object  in  the  image,  the  error  is  also  written  as  a  ratio  of  the  approximate  height 
of  the  image,  500  mm.  Therefore,  for  this  test,  pixel  resolution  is  1:1000,  and  millimeter 
resolution  is  1:500. 

Comparisons  were  made  at  three  zoom  and  focus  settings  within  the  DTM  envelope 
shown  in  Figure  3.12.  At  each  setting,  a  set  of  images  were  taken  of  the  calibration  grid 
like  the  one  shown  in  Figure  4.2  and  described  in  Table  4.1.  All  pictures  were  taken  with 
one  PTZ  camera.  The  calibration  grid  was  rotated  between  images  and  the  PTZ  camera 
was  held  fixed. 

The  vantage  points  for  the  images  for  lens  condition  one  is  shown  in  Figure  4.3. 
Condition  two  and  three  had  similar  configurations  at  increasing  distance,  except  at 

Table  4.1:  Data  on  the  calibration  grid  used  in  the  curve-fit  tests 

no.  of  points  144 

no.  of  coded  points  4 

length  of  side  536  mm 

approx,  scale  factor  0.5  mm/pixel 
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Figure  4.2:  PhotoModeler  calibration  grid.  8-12  images  of  the  grid  from  different 

angles  are  processed  to  generate  calibration  parameters  at  a  specific  lens  setting. 

condition  three  the  four  images  with  the  highest  angle  of  incidence  were  removed  because 
they  interfered  with  the  stability  of  the  calibration.  Twelve  images  were  used  at  the  short 
and  medium  focal  length,  and  eight  were  used  at  the  long  focal  length. 

Three  point  clouds  were  generated  at  each  lens  condition  using  the  same  image  set. 
All  points  were  subpixel  marked  in  each  image  for  maximum  accuracy.  The  first  point 
clond  was  generated  by  performing  a  full  calibration,  which  calculates  camera  parameters 
and  point  locations  for  the  grid.  The  full  calibration  allowed  PhotoModeler  to  decide 
which  camera  parameters  to  include  to  give  the  best  result.  The  camera  parameters 
included  were  /,  k±,  xq,  and  yo.  The  second  point  clond  was  generated  with  a  partial 
calibration  which  held  fixed  the  principal  point  and  solved  for  /,  k±,  and  k?.  The  third 
point  cloud  was  generated  using  curve-fit  calibration  parameters  and  the  same  principal 
point  as  the  partial  calibration.  The  value  at  which  the  principal  point  was  fixed  was  the 
average  of  three  values  from  prior  calibrations  at  the  zoom  settings.  The  three  sets  of 
points  will  be  referred  to  as  the  full  calibration  (FC)  point  clond,  the  partial  calibration 
(PC)  point  cloud,  and  curve-fit  parameter  (CP)  point  cloud. 

Tables  4.2,  4.3,  and  4.4  show  the  results  at  the  three  zoom  settings.  The  first 
thing  to  notice  is  that  the  PC  point  clond  had  similar  and  sometimes  better  residual  and 
tightness  than  the  FC  point  clond.  This  is  an  encouraging  result,  since  it  indicates  that 
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Figure  4.3:  Approximate  camera  locations  for  lens  condition  one  and  two.  Lens  con¬ 
dition  three  did  not  have  the  four  locations  with  highest  angle  of  incidence. 

the  upper  limit  of  the  internal  consistency  of  the  reduced  camera  model  is  as  good  as 
that  of  the  full  camera  model.  Internal  consistency  is  therefore  solely  a  function  of  the 
quality  of  the  curves  that  are  fit  to  the  parameter  values. 

Accuracy  of  the  PC  point  clouds  decrease  with  increasing  focal  length,  but  since  the 
full  and  partial  calibrations  are  of  similar  internal  consistency,  it  is  questionable  whether 
the  full  calibration  produced  significantly  more  accurate  point  locations  to  be  considered 
“truth” .  Only  large  differences  in  the  comparison  of  tightness  value  between  point  clouds 
can  be  considered  an  indication  of  relative  accuracy.  An  independent  measurement  of 
the  point  locations  should  be  considered  for  a  more  reliable  comparison  of  accuracy. 

The  CP  point  cloud  showed  a  significant  decrease  in  tightness  compared  to  the  PC 
and  FC  point  clouds.  Accuracy  was  on  the  order  of  a  pixel.  This  is  a  relatively  low  value 
for  point  accuracy  in  comparison  with  general  photogrammetry,  (see  Reference  [10]), 
although  it  would  still  be  acceptable  in  certain  cases.  The  low  accuracy  and  tightness 
is  most  likely  caused  by  a  poor  focal  length  curve-fit.  This  is  indicated  by  the  fact  that 
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the  tightness  of  the  CP  point  cloud  improved  at  higher  zoom  setting  and  lower  percent 
difference  between  the  curve-fit  and  calibrated  focal  lengths. 

As  further  evidence  of  the  need  to  improve  the  focal  length  curve,  consider  the 
residual  directions  for  the  images  used  for  processing  lens  condition  one.  Figure  4.4 
shows  the  residuals  for  the  PC  point  cloud,  which  do  not  exhibit  a  strong  pattern  that 
would  indicate  poor  modeling.  Compare  that  to  Figure  4.5.  The  images  with  lower 
incidence  angles  (image  9,  10,  11,  and  12,  the  bottom  row)  clearly  show  that  point 
calculations  for  those  images  were  less  trapazoidal  than  the  overall  point  locations.  This 
is  caused  by  the  camera  having  been  calculated  to  be  farther  from  the  grid  of  points  than 
it  actually  was,  which  in  turn  caused  it  to  expect  less  of  a  perspective  angle  than  was 
actually  the  case.  The  distance  of  the  camera  is  calculated  from  the  focal  length,  and  a 
larger  focal  length  is  equivalent  to  larger  distance. 

The  poor  focal  length  curve  fit  is  a  correctable  problem.  Additional  calibrations  can 
be  taken  with  special  attention  to  the  consistency  of  the  parameters  at  each  zoom  setting. 
Instead  of  weighting  the  calibration  results,  fewer  calibrations  should  be  performed  with 
a  emphasis  on  low  residuals  and  tightness.  The  performance  of  the  partial  calibration 
indicates  that  high  accuracy  and  tightness  are  possible  with  this  camera  model,  and  an 
improvement  of  the  focal  length  curve-fit  will  be  a  step  in  the  right  direction.  Though  the 
current  curve-fits  are  not  optimal,  the  results  of  this  test  indicate  that  this  is  a  suitable 
approach  for  calibrating  the  PTZ  camera  system  in  a  dynamic  environment. 

The  previous  test  showed  the  accuracy  of  the  calibration  curve  fits  under  a  con¬ 
trolled  situation  with  twelve  images.  The  test  shows  the  upper  limits  of  accuracy  with 
the  current  system.  An  actual  image  capture,  however,  will  be  taken  with  multiple  cam¬ 
eras,  and  will  have  only  four  images  to  process.  The  image  viewpoints,  which  have  a 
significant  impact  on  point  accuracy  and  precision,  will  not  be  able  to  be  controlled. 

To  demonstrate  a  photogrammetry  project  under  these  circumstances,  the  follow¬ 
ing  example  was  performed.  A  flat  board  was  covered  with  coded  photogrammetry 
targets  and  four  point  clouds  were  generated  using  calibrated  cameras  and  good  viewing 
angles.  These  point  locations  were  averaged  and  considered  truth.  The  truth  surface 
was  compared  to  four  additional  point  clouds.  The  comparison  clouds  were  generated 
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Figure  4.4:  Residual  direction  and  relative  scale  of  each  point  in  each  image  for  PC 

point  cloud  in  test  one 


Figure  4.5:  Residual  direction  and  relative  scale  of  each  point  in  each  image  for  CF 

point  cloud  in  test  one 
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Table  4.2:  Comparison  of  point  cloud  data  at  lens  condition  1 


Lens  Condition  1 

Zoom  Setting  -  25700,  Focus  Setting  -  21673 


Full  Calibration  Partial  Calibration  Curve-fit  Parameters 


/  (mm) 

27.8874 

27.9101 

32.65 

Camera 

k\ 

-2.059e-3 

-2.097e-3 

-2.228e-3 

k2 

-2.487e-5 

-3.855e-6 

2.852e-6 

Parameters 

x0 

2.9733 

2.9772 

2.9772 

yo 

2.4870 

2.4802 

2.4802 

RMS  residual  (pixels) 

.0867 

.0878 

.87 

Tightness 

mm 

.131 

.128 

1.38 

ratio 

1:4000 

1:4,000 

1:400 

RMS  error 

mm 

.0306 

.595 

ratio 

1:18,000 

1:900 

RMS  error 

X 

.00927 

.0117 

.0927 

.145 

Directional 

y 

.0267 

.569 

z 

Table  4.3: 

Comparison  of  point  cloud  data  at  lens  condition  2 

Lens  Condition  2 

Zoom  Setting  -  31993,  Focus  Setting  -  27349 

Full  Calibration 

Partial  Calibration 

Curve-fit  Parameters 

/  (mm 

)  37.8998 

37.9645 

41.2521 

Camera 

k\ 

-2.112e-3 

-2.102e-3 

-1.966e-3 

k2 

1.870e-5 

1.623e-5 

6.991e-6 

Parameters 

x0 

2.981 

2.9772 

2.9772 

yo 

2.4821 

2.4802 

2.4802 

RMS  residual  (pixels) 

.1213 

.1034 

.41 

Tightness 

mm 

.186 

.1456 

.618 

ratio 

1:3000 

1:4,000 

1:900 

RMS  error 

mm 

.0700 

.284 

ratio 

1:8,000 

1:2,000 

RMS  error 
Directional 

X 

y 

.0108 

.0123 

.0737 

.0794 

z 

.0680 

.190 
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Table  4.4:  Comparison  of  point  cloud  data  at  lens  condition  3 


Lens  Condition  3 

Zoom  Setting  -  38702,  Focus  Setting  -  33026 

Full  Calibration 

Partial  Calibration 

Curve-fit  Parameters 

/  (mm) 

52.4612 

52.6104 

56.6318 

Camera 

k\ 

-2.053e-3 

-1.997e-3 

-1.664e-3 

k2 

1.431e-5 

1.442e-5 

1.22e-5 

Parameters 

x0 

2.9758 

2.9772 

2.9772 

yo 

2.4778 

2.4802 

2.4802 

RMS  residual  (pixels) 

.1213 

.1089 

.30 

Tightness 

mm 

.189 

.1319 

.3542 

ratio 

1:3000 

1:4,000 

1:1,500 

RMS  error 

mm 

.1542 

.7294 

ratio 

1:4,000 

1:700 

RMS  error 

X 

.0682 

.1277 

.495 

.449 

Directional 

y 

.0530 

.1949 

z 

using  all  four  PTZ  cameras  on  the  outside  of  the  room  and  the  photogrammetry  object 
placed  in  the  center.  Four  images  of  the  object  were  taken  for  each  point  cloud,  and 
the  object  was  rotated  90°  between  each  set.  Each  set  was  processed  using  curve  fit 
calibration  parameters  and  the  error  for  each  point  is  shown  in  Figure  4.6.  In  order  to 
observe  internal  error  alone,  the  point  clouds  were  corrected  for  errors  due  to  translation, 
rotation  and  scaling  by  minimizing  the  error  with  respect  to  those  parameters,  and  the 
corrected  error  values  also  are  also  shown  in  Figure  4.6.  These  corrections  were  small, 
with  translations,  rotations,  and  scale  factors  less  than  0.5  mm,  1°,  and  1%  respectively. 
Note  that  the  error  in  positions  1  and  3  are  approximately  double  that  of  2  and  4,  and 
since  the  object  was  rotated  90°  between  positions,  positions  1  and  3  were  in  similar  but 
reverse  orientations.  In  these  positions,  the  object  was  in  a  less  favorable  orientation, 
with  the  majority  of  the  markers  at  a  shallower  angle  than  in  positions  1  and  4.  The 
transformed  error  normalized  by  the  object  length  is  approximately  1:400  for  positions 
2  and  4,  and  1:200  for  positions  1  and  3.  Thus  it  can  be  see  that  image  orientation  of 
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the  images  can  have  a  negative  effect  on  the  point  cloud  accuracy.  More  information  on 
the  effects  of  object  orientation  and  can  be  found  in  reference  [6]. 

The  results  of  the  curve  fit  calibration  parameters  test  indicate  the  viability  of  using 
even  very  basic  curve-fitting  for  generating  calibration  parameters.  Point  clouds  gener¬ 
ated  under  operational  conditions  show  additional  loss  of  accuracy,  but  it  is  supposed 
that  this  will  improve  with  improved  curve  fits.  Refinement  of  the  curve  fit  calibra¬ 
tions  will  be  the  subject  of  future  work.  The  use  of  this  method  allows  the  generation 
of  calibration  parameters  on  a  continuous  scale,  and  is  key  to  enabling  the  use  of  the 
pan-tilt-zoom  cameras  for  dynamic  photogrammetry. 

4-4  Validation  of  Image  Synchronization 

For  dynamic  reconstruction  of  objects  in  motion  to  be  accurate,  it  is  important 
that  each  set  of  four  images  be  synchronized.  The  synchronization  system  described 
above  was  validated  with  the  following  test.  A  sequence  of  images  was  captured  of  a 
small  direct  current  electric  motor  running  at  approximately  11  revolutions  per  second. 
Attached  to  the  motor  is  a  paper  “fan”  with  a  drawing  on  one  side,  so  that  its  orientation 
can  be  determined.  The  trigger  signal  was  run  at  500hz  and  the  cameras  were  set  up  to 
capture  images  on  the  rising  edge  of  the  signal.  The  four  images  in  Figure  4.7  are  image 
352  of  each  of  the  sequences,  and  although  the  fan  image  is  not  sharp  due  to  the  exposure 
time,  the  darker  area  caused  by  the  drawing  on  the  left  side  shows  that  it  has  the  same 
orientation  in  each  of  the  images.  This  assures  that  the  system  has  the  capability  of 
synchronous  image  capture  to  within  one  frame.  However,  by  analyzing  the  blur  of  the 
fan  tip  and  noting  its  beginning  and  end  in  each  photo,  it  appears  to  be  even  better  than 
that. 

4-5  Validation  of  Tracking  System 

The  tracking  system  has  two  parts:  the  camera  aiming,  accomplished  by  the  gimbal, 
and  the  focus  and  zoom,  controlled  by  the  lens.  Each  are  controlled  with  separate  loops 
in  the  PTZ  controller  software  as  described  in  section  3.6,  and  they  both  pose  limits  on 
the  ability  of  the  PTZ  cameras  to  produce  suitable  images  for  photogrammetry.  The 
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Position  1  Error  Position  2  Error 


Distance  from  origin  (mm)  Distance  from  origin  (mm) 

Figure  4.6:  Error  for  four  photogrammetry  projects  compared  to  the  truth  surface. 

Circles  denote  raw  points  and  triangles  denote  points  scaled,  translated  and  rotated  for 
optimum  fit.  Points  are  plotted  against  scalar  distance  from  origin.  The  object  is  rotated 
90°  between  each  position. 
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(a)  Camera  1  image  (b)  Camera  2  image  (c)  Camera  3  image  (d)  Camera  4  image 

Figure  4.7:  The  four  high-speed  cameras  capturing  synchronous  images  at  500fps.  Note 
that  the  spinning  paper  “fan”  has  the  same  orientation  with  respect  to  the  mounting 
platform  in  each  image. 

following  section  describes  the  tests  run  to  evaluate  the  capabilities  of  the  girnbal  and 
its  effects  on  camera  aiming.  The  lens  analysis  was  not  performed  for  this  thesis. 

Total  camera  aiming  error  is  the  result  of  several  factors.  Static  pointing  error 
results  from  approximations  made  in  the  aiming  algorithm  and  camera  location  and  ori¬ 
entation  error.  Dynamic  error  is  mostly  caused  by  the  girnbal  dynamics,  with  additional 
dynamic  error  caused  by  time  lag  from  the  videogrammetry  system,  control  software, 
and  serial  communication.  As  a  first  step  in  determining  and  reducing  the  error  caused 
by  each  of  these  factors,  the  total  aiming  error  was  evaluated. 

Initial  validation  of  the  aiming  system  was  performed  by  tasking  the  camera  to 
track  a  videogrammetry  object  attached  to  the  rotation  table  in  Figure  3.4.  The  ob¬ 
ject  maintained  its  orientation  as  it  spun,  and  four  coded  photogrammetry  targets  were 
placed  on  the  side  facing  the  camera.  The  coded  targets  are  round  markings  that  are 
automatically  marked  and  referenced  between  frames  by  the  photogrammetry  software. 
One  target  was  placed  at  the  aiming  point,  and  the  others  were  placed  near  the  x  and  y 
axis  limits  of  the  object  for  scaling  purposes. 

The  regular  motion  of  the  rotation  table  allowed  the  position  and  orientation  of  the 
tracked  object  to  be  known  in  each  frame  without  synchronizing  the  image  capture  with 
the  videogrammetry  system.  This  allowed  any  delay  in  the  processing  of  the  videogram¬ 
metry  data  to  be  included  in  the  error  assessment.  The  radius  and  center  of  the  circular 
path  was  determined  by  the  least  squared  error  of  the  center  to  the  position  minus  the 


53 


radius  squared 


ecf  =  -  x)2  +  (y*  ~  y )2  - f2)2  (4-4) 

i=  1 

where  ecf  is  the  circle  fit  error,  Xi  and  yl  are  positions  generated  by  the  videogrammetry 
system,  and  x,  y,  and  r  are  the  center  location  and  radius.  The  radius  r  can  be  written 
in  terms  of  the  other  variables 


r  = 


\ 


n 


^((xi  -  x)2  +  (y.i  -  yf 


r  /  0 


(4.5) 


i=  1 


and  using  the  notation  x  to  designate  the  mean,  x  and  y  can  be  solved  for  from  two 
simultaneous  equations 


2 

EIE  (xi(xi  -  x ))  EIE  (xi(yi  -  y)) 

X 

EILi  (x2(xi  -x)  +  y2(xi  -  x)) 

EILi  Mi-v))  EILi  iviivi-v)) 

y 

EILi  (x?(yi~y)  +  yKyi-y)) 

(4.6) 


The  turntable  was  rotated  at  a  constant  speed  using  motors  and  a  belt  drive,  and  an 
LED1  was  triggered  to  flash  when  the  object  passed  a  certain  point  in  the  rotation.  The 
LED  flash  could  be  seen  in  the  PTZ  camera  as  it  tracked,  which  allowed  the  period  and 
phase  of  the  rotation  to  be  determined  directly  from  the  images.  The  period,  phase,  and 
radius  of  the  rotation  table  was  used  to  generate  the  function  for  the  continuous  periodic 
truth  input  to  which  the  tracking  error  and  output  was  compared. 

Fifteen  tracking  tests  were  run;  six  rotating  the  table  in  each  direction  at  a  distance 
of  2.5  m  and  speeds  ranging  from  0.2  m/s  to  0.9  m/s,  and  three  at  approximately  0.55 
m/s  and  increasing  distance  to  a  maximum  of  4.25  m.  During  each  test  the  PTZ  camera 
captured  a  sequence  of  images  at  30  frames  per  second  and  in  each  image  the  distance 
between  the  target  point  and  the  image  center  was  measured  in  pixels,  as  shown  in  Figure 
4.8.  The  additional  scaling  targets  were  used  to  determine  a  pixel  to  mm  scale  factor 

1  light-emitting  diode 
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in  each  direction  for  each  image.  The  millimeter  error  summed  with  the  truth  function 
produced  the  output  of  the  camera  system. 

The  motion  of  the  object  was  separated  into  its  x  and  y  components,  with  x 
component  of  motion  tracked  by  the  azimuth  servo  of  the  gimbal  and  y  component  by 
the  elevation  servo.  The  millimeter  error  in  each  direction  was  measured  and  Figure 
4.9  shows  the  results  plotted  versus  object  speed  in  meters  per  second.  RMS  values 
in  the  y  direction  are  lower  than  the  x  because  the  angle  of  view  causes  the  velocity 
perpendicular  to  the  camera  vector  to  be  reduced.  Test  points  marked  by  diamonds 
were  run  at  distances  greater  than  the  2.5  m,  with  the  farthest  distance  being  4.25  m. 

The  fact  that  the  diamond-marked  points  lie  on  the  same  curve  as  the  other  points 
indicate  a  minimal  dependence  of  the  millimeter  tracking  error  on  distance.  This  is 
because  of  the  counterbalancing  of  the  effects  of  angle  error  and  angular  velocity  of  the 
tracked  object  about  the  gimbal.  At  close  range,  the  gimbal  is  required  to  rotate  faster 
in  order  to  track  an  object  at  a  given  speed  than  at  a  farther  distance,  thus  causing  a 
larger  angular  error.  However,  a  large  angular  error  at  close  range  results  in  a  smaller 
linear  error  at  the  object  than  when  further  away.  Thus,  the  two  effects  cancel  each 
other.  Additional  testing  may  show  that  the  cancellation  is  not  perfect,  especially  at 
extremely  close  and  far  distances,  but  in  the  lab  capture  volume  it  holds  well. 

The  RMS  angular  error  of  the  gimbal  was  found  using  the  center  of  the  rotation 
table  as  the  average  distance  to  the  object.  The  RMS  angular  velocity  of  the  gimbal 
required  by  the  speed  of  the  rotation  table  was  calculated  as  shown  in  Equations  4.7  and 
4.8 


toaz  =  V2vd  (4.7) 

u>ei  =  \/2sin  (4>)vd  (4.8) 

where  v  is  the  speed  of  the  rotation  table,  d  is  the  distance  from  the  gimbal  to  the  center 
of  the  rotation  table,  and  (f)  is  the  camera  angle  of  incidence.  The  angular  error  was 
plotted  against  the  angular  velocity  and  is  shown  in  Figure  4.10 
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Figure  4.8:  Camera  aiming  error  is  measured  in  pixels  from  the  center  image  pixel  to 
the  coded  target  on  the  box. 
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(a)  RMS  mm  x  axis  error  vs  object  speed  (b)  RMS  mm  y  axis  error  vs  object  speed 


Figure  4.9:  RMS  mm  error  vs  speed  of  rotation  table.  Points  marked  with  x’s  were 

tested  at  2.5  m  from  camera.  Diamond  points  indicate  tests  run  at  increased  distances. 


Two  notes  should  be  made  about  the  data.  There  is  clearly  more  angular  error  in 
the  elevation  axis  than  the  azimuth  axis  shown  in  Figure  4.10(b).  This  is  most  likely 
caused  by  the  camera  not  being  perfectly  balanced  on  the  gimbal,  so  that  the  gimbal  has 
to  overcome  the  effects  gravity  in  addition  to  inertia  when  rotating  to  point  downwards. 
The  additional  moment  required  by  the  gimbal  in  that  direction  slows  the  overall  response 
time.  Secondly,  the  drop  in  error  for  the  final  data  point  in  Figures  4.9(a)  and  4.10(a) 
are  due  to  data  dropout  rather  than  a  true  reduction  in  error.  At  the  peak  amplitude 
and  the  highest  turntable  speed,  the  scaling  targets  were  outside  the  field  of  view  of 
the  camera,  and  so  no  millimeter  error  could  be  calculated.  When  the  RMS  error  was 
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(c)  Combined  azimuth  and  elevation  error 


Figure  4.10:  RMS  Angular  error  vs.  RMS  angular  velocity  of  gimbal.  Points  marked 
with  x’s  were  tested  at  2.5  m  from  camera.  Diamond  points  indicate  tests  run  at  increased 
distances. 


calculated  for  that  point,  the  lack  of  the  highest  error  data  artihcally  reduced  the  RMS 
value.  Nonetheless,  this  point  was  included  for  completeness,  as  it  is  included  in  the 
following  phase  shift  analysis  (where  the  data  dropout  does  not  effect  the  result). 

The  phase  shift  and  magnitude  change  were  calculated  by  fitting  the  output  data 
to  sinusoidal  curves  using  a  least  squares  method  and  then  comparing  its  magnitude  and 
phase  to  the  truth  input.  Figure  4.11  show  the  resulting  Bode  plots. 

The  Bode  plots  will  provide  a  valuable  data  source  for  modeling  of  the  system 
dynamics.  Future  efforts  to  improve  the  aiming  performance,  such  as  adding  a  Kalman 
filter,  will  require  a  model  of  the  system. 
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Increasing  speed  has  the  effect  of  reducing  the  resolution  of  the  cameras  by  limiting 
how  small  a  field  of  view  can  be  used.  For  the  ideal  case,  with  perfect  zoom  and  focusing 
such  that  the  object  remains  the  same  size  in  all  images,  the  number  of  pixels  per  unit 
length  would  be  fixed,  and  the  value  would  depend  on  the  anticipated  speed  of  the 
object.  The  object  resolution,  n0  pixels/object  radius,  is  related  to  the  maximum  speed 
and  radius  of  the  object  in  the  ideal  case  by 


Tin  = 


rip/ 2 


T'obj  T  t’rnrn  (  l 


T'obj 


(4.9) 


where  np  is  the  number  of  pixels  across  the  shortest  dimension  of  the  image  plane,  r0bj  is 
the  half  the  largest  dimension  of  the  object,  and  emm(v)  is  the  RMS  millimeter  tracking 
error  at  speed  v.  Using  current  laboratory  cameras  with  1280x1024  resolution,  and  a 
typical  object  radius  of  15  cm,  the  plot  in  Figure  4.12(a)  was  constructed.  The  plot 
indicates  an  expected  decrease  in  resolution  of  the  object  speed  increases.  A  second  plot 
in  Figure  4.12(b)  shows  the  object  resolution  plotted  vs.  length  of  the  object,  and  shows 
that  larger  objects  can  be  covered  with  more  pixels  than  smaller  ones,  even  though  the 
pixel/unit  length  value  may  decrease. 

The  camera  aiming  system  described  above  is  capable  of  keeping  vehicles  in  the 
frame  under  the  given  restrictions  of  speed  and  resolution.  To  take  a  typical  example,  an 
object  of  15  cm  radius  with  a  desired  pixel  resolution  of  1.7  pixel/mm  would  be  limited 
to  speeds  less  than  0.9  m/s.  The  system  generally  favors  large,  slow  moving  vehicles, 
because  it  is  able  to  cover  these  vehicles  with  more  pixels  per  object  size.  There  are 
several  ways  in  which  this  system  could  be  improved.  One  is  the  addition  of  a  Kalman 
filter,  which  uses  predicted  future  position  of  the  object  to  improve  aiming  accuracy. 
Another  possibility  is  to  use  a  custom  girnbal  controller  that  puts  a  greater  premium  on 
error  reduction,  at  the  expense  of  such  effects  as  overshoot.  The  girnbal  is  capable  of 
using  a  custom  controller,  which  is  important  for  continuing  development  of  the  tracking 
system. 
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4-6  Tracking  Example:  Helicopter 


To  demonstrate  the  tracking  system,  a  sequence  of  images  was  captured  of  a  MAV 
in  flight.  The  MAV  is  a  radio-controlled  helicopter  with  two  couterrotating  rotors  and  two 
smaller  maneuvering  rotors  shown  in  Figure  4.13.  The  helicopter  was  marked  with  6  mm 
retroreflective  videogrammetry  targets  and  defined  as  an  object  in  the  videogrammetry 
software.  All  four  PTZ  cameras  were  set  to  track  and  image  the  helicopter  as  it  was  flown. 
The  field  of  view  was  set  in  the  PTZ  controller  at  0.5  m.  Nine  hundred  synchronous 
images  were  taken  at  30  fps  with  a  shutter  speed  of  5  ms,  for  a  total  capture  time  of  30 
seconds. 

The  location  of  the  helicopter  was  recorded  during  image  capture.  The  location 
data  at  several  time  steps  is  missing,  however,  because  at  those  time  steps  the  videogram¬ 
metry  system  had  lost  the  helicopter.  This  occurs  at  only  a  small  number  of  locations, 
and  the  gaps  are  clearly  visible  in  the  location  and  speed  plots.  This  problem  was  caused 
by  using  a  calibration  of  the  videogrammetry  system  that  was  several  days  old.  A  new 
calibration  would  reduce  and  possibly  eliminate  data  dropout. 

The  x  and  y  location  of  the  helicopter  is  shown  in  Figure  4.14,  and  the  z  location 
is  plotted  with  respect  to  image  number  in  Figure  4.15(a).  A  rough  plot  of  the  instanta¬ 
neous  speed  of  the  the  helicopter  is  given  in  Figure  4.15(b).  The  speed  was  calculated  by 
taking  the  backwards-difference  of  the  position  measurements  and  dividing  by  the  time 
interval,  shown  in  Equation  4.10. 


Vi  = 


I  Ti>  Vii  A  1?  Vi—h  A-l.  I 


i—  1 


(4.10) 


The  time  interval  was  varied  until  a  value  was  found  that  appeared  to  be  a  good  balance 
between  resolution  and  noise  reduction. 

The  helicopter  began  on  the  ground,  and  was  flown  smoothly  to  altitude.  At 
approximately  image  510  the  helicopter  lost  altitude  rapidly  and  hit  the  ground,  then 
recovered  after  a  low  sweeping  turn.  This  area  is  circled  in  Figure  4.14  and  the  drop  and 
speed  increase  is  clearly  visible  in  Figures  4.15(a)  and  4.15(b).  The  helicopter  recovered 
altitude  slowly  and  the  capture  ends  in  mid-flight. 
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Figures  4.16  and  4.17  show  an  excerpt  of  30  images  from  the  900  image  sequence 
from  camera  four.  The  images  are  at  one-second  intervals  and  the  image  number  is 
listed  below  each.  During  the  initial  ascent,  with  speed  peaking  at  about  0.8  m/s,  the 
helicopter  remains  in  view  of  the  camera.  At  image  540,  the  helicopter  is  completely 
outside  the  camera  frame,  and  does  not  fully  recover  until  image  around  600.  The  loss 
of  the  vehicle  occurs  during  the  high-speed  descent,  when  a  speed  of  approximately  1.8 
m/s  is  reached. 

The  helicopter  remains  in  focus  until  the  end  of  the  sequence,  around  frame  780 
to  810.  As  this  occurs  while  the  helicopter  is  moving  relatively  slowly  away  from  the 
camera,  the  lens  is  likely  the  limiting  factor  in  the  speed  at  which  objects  can  be  tracked. 
A  thorough  analysis  of  the  lens  is  needed  to  determine  what  that  limit  is. 

This  example  shows  the  ability  of  the  tracking  system  to  maintain  focus  and  framing 
of  a  flying  vehicle.  The  performance  of  the  system  during  the  capture  is  shown  to 
generally  match  the  predicted  performance  in  the  previous  section.  This  system  will 
provide  an  effective  means  of  gathering  images  suitable  for  photogrammetry  in  a  dynamic 
environment,  a  unique  capability. 
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Figure  4.11: 
from  camera. 


Bode  plots  of  tracking  data.  Points  marked  with  x’s  were  tested  at  2.5  m 
Diamond  points  indicate  tests  run  at  increased  distances. 
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Figure  4.12:  Resolution  as  a  function  of  object  speed  and  size,  assuming  ideal  lens 

operation. 


Figure  4.13:  Twin  rotor  helicopter  used  in  example  capture 
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Figure  4.14:  Helicopter  and  camera  x-y  locations  during  image  capture 


Figure  4.15:  Helicopter  speed  and  height  during  image  capture 
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V.  Conclusions  and  Future  Work 


5. 1  Conclusion 


This  thesis  developed  and  evaluated  a  hybrid  DTM  system  for  collecting  kinematic 
measurements  of  structural  surfaces  in  motion.  The  system  will  provide  unique 
data  sets  on  objects  such  as  MAVs  in  flight  and  deployable  space  structures  that  will  aid 
in  the  design  and  development  of  these  systems. 

The  system  components  were  described,  and  details  were  given  on  the  control  and 
integration  of  the  cameras.  Calculation  of  the  azimuth  and  elevation  angles  for  the 
gimbals  was  presented.  Multiple  methods  of  locating  the  cameras  were  presented  and 
compared,  and  location  by  the  videogrammetry  system  was  shown  to  be  the  preferred 
method  because  of  its  ease  of  use  and  robustness.  The  method  of  calculating  the  zoom 
and  focus  settings  to  maintain  a  constant  field  of  view  was  developed  and  the  envelope 
of  the  zoom  and  focus  was  outlined.  The  tracking  system  was  combined  and  controlled 
in  a  computer  program  and  details  of  its  operation  were  addressed. 

Several  test  of  the  system  were  performed  to  evaluate  performance  and  find  areas 
needing  improvement.  The  tests  were:  analysis  of  the  use  of  curve  fits  for  calibration  pa¬ 
rameters,  demonstration  of  camera  synchronization,  analysis  of  dynamic  camera  aiming 
error,  and  a  demonstration  of  the  system  tracking  and  taking  images  a  MAV  in  flight. 

Using  curve  fit  parameters  for  dynamic  photogrammetry  is  a  novel  technique.  The 
technique  was  tested  to  determine  if  it  would  produce  accurate  and  precise  point  clouds. 
Point  clouds  generated  with  curve  fit  calibration  parameters  were  compared  to  those  gen¬ 
erated  with  partial  and  full  calibrations.  The  resulting  curve  fit  point  clouds  were  found 
to  have  an  accuracy  around  1  unit  error  per  thousand  (1:1,1000),  which  is  equivalent 
to  pixel-sized  error,  and  RMS  residual  around  0.5  pixels.  The  focal  length  parameter 
generated  by  curve  fit  was  significantly  higher  than  those  generated  by  partial  and  full 
calibrations,  and  analysis  of  the  residual  directions  indicates  that  the  focal  length  curve 
fit  is  the  primary  source  of  point  error  and  low  precision.  The  reduced  camera  model 
that  was  used  was  shown  to  have  comparable  precision  to  the  fully  calibrated  cameras. 
This  result  indicated  that  with  improvement  of  the  curve  fits,  especially  the  focal  length 
curve  fit,  accuracy  of  around  1:10,000  can  be  expected. 
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Camera  synchronization  was  established  by  taking  a  capture  with  all  four  cameras 
at  500  fps  of  a  rotating  motor.  The  motor  was  shown  to  be  in  the  identical  orientation 
in  each  camera,  indicating  that  all  four  images  were  taken  synchronously. 

The  dynamic  camera  aiming  error  was  evaluated  and  found  to  be  within  reasonable 
limits.  The  dependence  of  maintaining  view  of  an  object  on  the  object  speed  and  the  field 
of  view  was  described,  and  the  relationship  between  object  speed  and  allowable  object 
resolution  was  presented.  It  was  shown  that  speeds  approaching  0.9  m/s  are  attainable 
with  1.7  pixel/mm  resolution  and  15  cm  object.  As  an  example  of  the  tracking  system 
in  operation,  a  sequence  of  images  of  a  MAV  flying  in  the  lab  was  shown.  The  system 
was  able  to  frame  and  focus  on  the  vehicle  throughout  most  of  the  30  s  flight,  and  the 
performance  of  the  system  approximately  followed  the  predicted  performance. 

Based  on  the  results  shown  here,  this  photogrammetry-based  hybrid  DTM  system 
is  capable  of  tracking  and  recording  the  necessary  image  data  for  generating  surface 
profiles  of  lightweight  dynamic  structures.  The  hybrid  measurement  concept  that  this 
system  is  based  on  overcomes  the  limitations  of  the  individual  systems  and  allows  a  new 
look  at  the  behavior  of  lightweight  structures.  It  has  the  potential  to  be  expanded  by 
adding  additional  sensors  along  with  the  PTZ  cameras,  such  as  laser  vibrometers  and 
laser  range  finders.  In  conclusion,  the  measurement  system  presented  in  this  thesis  has 
the  capability  to  give  researchers  new  insight  into  lightweight  structures  in  a  natural 
environment,  and  the  possibilities  offered  by  the  hybrid  measurement  approach  are  only 
just  beginning  to  be  explored. 

5. 2  Future  Work 

Future  work  on  the  DTM  system  will  focus  on  finishing  the  characterization  of 
the  system  with  an  analysis  on  the  lens  and  the  limitations  it  places  on  object  tracking. 
Lens  analysis  will  include  determining  how  quickly  the  lens  can  adjust  zoom  and  focus, 
and  how  focus  error  affects  the  photogrammetric  processing.  Additional  improvement 
of  the  system  with  a  custom  designed  closed  loop  girnbal  controller  has  the  potential 
to  reduce  aiming  error.  Improvement  of  the  curve  fit  parameters  will  likely  increase 
photogrammetric  accuracy. 
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Since  the  purpose  of  the  system  is  the  photogrammetric  reconstruction  of  objects 
in  motion,  methods  of  automating  the  processing  of  image  data  are  needed.  Generating 
surface  profiles  is  currently  a  labor-intensive  process,  and  manual  processing  of  poten¬ 
tially  thousands  of  frames  of  image  data  is  not  practical.  Initial  steps  have  been  taken 
in  automating  the  processing  such  as  controlling  the  photogrammetry  software  with  a 
computer  script,  and  future  work  will  build  on  this  start. 
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videogrammetry  system,  a  set  of  pan-tilt-zoom  (PTZ)  cameras  is  coordinated  to  track  large-scale  net  motion  and 
produce  high  speed,  high  quality  images  for  photogrammetric  surface  reconstruction.  The  design  of  the  system  is 
presented  in  detail.  A  method  of  generating  the  calibration  parameters  for  the  PTZ  cameras  is  presented  and 
evaluated  and  is  shown  to  produce  good  results.  The  results  of  camera  synchronization  tests  and  tracking 
accuracy  evaluation  are  presented  as  well.  Finally,  a  demonstration  of  the  hybrid  system  is  presented  in  which  all 
four  PTZ  cameras  track  an  MAV  in  flight. 
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